Parameterise a IN clause in T-SQL

Parameters are a great feature in any SQL implementation. You can protect yourself – and more likely – your clients from SQL injection by always using parameterised queries for every call to the database.

I’ve seen some implementations in e-commerce packages where SQL injection would be a breeze.

The Scenario

For a recent client I had a .Net app which was to query records but exclude a set from a specific set of criteria. Normally this is either done with a join or an IN clause. In this instance as I am passing the criteria by parameter I needed a IN Clause.

A IN clause is used to either include or exclude records from a result. An example would be:

Select * from TableA where TableA.name IN ('John','Frank','Dave')

This would include only those rows where column name matched one of ‘John’ or ‘Frank’ or ‘Dave’.

The Problem

Now there is no current way to send a array of items through a parameter. So your choices are:

  • Invoke the query once for each criteria item, or
  • Use dynamic SQL

Both of these are highly inefficient, to say the least.

The Solution

Now before you start pulling your hair out, there is a simple and elegant solution. In T-SQL you can use XML, yes I did write XML that is not a typo.

Here’s how

In T-SQL you can have script which looks like:

Declare @x xml
@x = '<L>
        <I>John</I>
        <I>Frank</I>
        <I>Dave</I>
     </L>'

Select * from TableA where TableA.name in (
     Select * from ( select T.c.value('.','uniqueidentifier') as source
     from @x.nodes ('/L/I') T(c) ) Idss )

The inner query converts the XML into a result set by breaking out the individual <I> nodes within the <L> node into logical rows. The ‘T.c.value’ then accesses the logical row and extracts the text within. Finally this is presented as a subquery result named Idss (apparently that part is important otherwise you’ll get an error).

Now all you would need to do is to place your XML in the @x parameter. I chose a parameter type of string as that was simpler to handle.

Conclusion

The XML solution is quite elegant requiring minimal effort on the client side to pass the parameters to the server. This is much faster than dynamic SQL, as the server store the plan away and will save you recompiles on the server – always a performance killer.

You’ve seen how simple it is to specific a list of values for an IN clause as a parameter. A lot simpler than either of thought!

Posted in .Net, sql, SQL Server, Uncategorized | Tagged , , , , | Leave a comment

Alias Folders in Dovecot

Since I’ve now been able to use my mobile phone to send e-mails I have noticed a small issue. The folders which my Outlook client uses to store the sent e-mails is different than what the Phone client uses.

I noticed this issue when I went to look in the ‘Sent’ folder in Outlook and didn’t see the any of the e-mails which I knew I had sent from the phone. A quick look around revealed a new  folder ‘.sent items’ had been created and within it the e-mails were stored. arrgh.

I had a choice to make. I can either reconfigure Outlook and SquirrelMail to store e-mails in Sent Items – but then I thought – if others also started using Phones I would have to alter their configurations. Probably a lot of unnecessary work.

Unfortunately in dovecot – my chosen IMAP server – there is no aliasing of folders capability. The only way to  get this to work is  to use filesystem symlinks (symbolic links).

Once I’d moved the e-mails into the Sent folder I then ‘fixed’ this with

sudo rm -r ".sent items"
sudo ln -s .Sent ".sent items" 

And similarly my client also used ‘.deleted items’ so I linked .deleted items to .Trash

This is truly ugly, but thre is now other way to do this right now.

Posted in Uncategorized | Tagged , , | Leave a comment

Hooking WP7 to exim4 and dovecot with SSL

Okay, so I go this new phone which allows me to access my gmail and hotmail accounts – I have a lot of e-mails. Part of the problem with my frazzled brain is that I setup a lot of different e-mails

Setting up Dovecot

Dovecot was remarkably simple to setup and get the authentication working. A quick follow of these instructions got the SSL connection to my phone working. I was even able to reuse my self-signed certificate which I use for other services on the same CNAME.

Setting up Exim SSL

Exim has two levels here. One is TLS/SSL setup and other is authentication. I first started with SSL to encrypt the content and authentication information. Seems obvious doesn’t it?

My system is a Ubuntu(Debian) system using the single file configuration. I find this far easier to manage.

This is where I ran into my first issue. A follow of these instructions did not fully enable the  connection with the phone.  The Exim log indicated repeated problems with the TLS connection. Not so easy after all.

Reasearch indicates that some e-mail clients are unable to use the new STARTTLS syntax and instead used a immediate jump into SSL. I would be miffed if that was the case with the Windows Phone client. I set about trying it anyway.

Exim has the setting

 tls_on_connect_ports = 465 

This initiates the SSL connection from the start. This did not work either giving other errors. It appeared that my SSL certificates were not compatible with Exim – even though they worked fine with dovecot. (see above)

However even after I used the certificate generation tool

 exim-gencert 

I still received the same errors. This was beyond my meagre skills. So I moved on, perhaps another day I will find the answer.

Setting up Exim Auth

This part of the setup was very much easier, but sorting out the Debian single file setup was a bit of fun. Not really.

As I use LDAP for authentication and mapping of the virtual e-mail addresses the basic Exim – Debian setup was required to be changed. Below is what I used and is pretty self explanatory.

plain:
 driver = plaintext
 public_name = PLAIN
 server_condition = ${if ldapauth{user="uid=${quote_ldap_dn:$2},PEOPLE_BASEDN" pass=${quote:$3} \
 ldap://localhost/} {yes}{no}}
 server_set_id = $2

login:
 driver = plaintext
 public_name = LOGIN
 server_prompts = Username:: : Password::
 server_condition = ${if and{ {!eq{}{$1} }\
 {ldapauth{user="uid=${quote_ldap_dn:$1},PEOPLE_BASEDN" pass=${quote:$2} \
 ldap://localhost/} }} \
 {yes}{no}}
 server_set_id = $1 

Note that the PLAIN authenticator does not have prompts and that the userid is $2, because $1 is and unique ID passed through, but often not used. As well there is no empty check for userid – like there is with the LOGIN. This seemed to cause errors.

Summary

I can now read and send e-mails from my phone which is great. While the send cannot use SSL for now, this is something I hope to find an answer for.

One last issue is that the phone client likes to store outgoing e-mails in the Sent Items folder and trash in Deleted Items. This is not the standard setup that Outlook and client use so I will show you how to fix that in my next post.

Posted in Exim, exim4, IMAP, ldap, Linux, ssl, Uncategorized | Tagged , , , , , | Leave a comment

My new phone

Yay!

I finally broke down and – even though I could not afford it – I bought a new phone for myself. I got a LG-E906, branded as a Jil Sander, a Windows Phone mobile phone.

Not having had a phone like this in the past I’m quite impressed with what it can do. I easily hooked it up to my WiFi connection and was able to get at my gmail and hotmail and other social accounts really easy.

I don’t know what people with iPhone and Android phones experience, but this is cool. The angry birds game is a hit with my son too.

Posted in Uncategorized | Tagged | Leave a comment

This Worked

Yesterday I could not believe what I did.

I created a new WordPress site for my self, for my freelancing business. After pretending I was an agency (an agency of one hah!) I have finally woken up. I am a freelancer and that how I sell myself now. I quickly changed my LinkedIn profile and bought some domains. And now I put this site up on my hosting.

This site was originally developed on my laptop and transferred to my hosting. This is how I did it, which doesn’t mean that it works for everyone.

I zipped up all the files and loaded and unzipped the files to the folder for the domain on my hosting. then I created a export file for the mySQL database. I had to modify it slightly to remove the CREATE DATABASE line at the top and put a USING <dbname>.

I entered my cpanel on my hosting and created the database and user. Then input these credentials and database name into the wp-config.php file.

Upload the databse using the phpmysqladmin tool. Then edit the table <prefix>options and change the siteurl and home elements (filter on the option_name column) to match the url to wordpress site.

And it worked!…well almost. The URL rewriting didn’t work. But this can be ‘kicked’ by going to the permalinks settings and clicking save.

Then, by god, it worked. And you are now reading this blog.

Posted in business, CMS, Uncategorized | Tagged , | Leave a comment

How to refuse users to send mail

In certain circumstances I have found the need to prevent certain user’s from sending e-mail outside my domain.

These user’s are mainly my kids. Who enjoy sending people e-mail – with addresses they can’t replay to (more later).

The way I have my e-mail setup is that I have a host which collects my e-mails at my domain addresses. And I have a fetchmail process which collects this e-mail and drops it into the appropriate folders

But sending e-mail I chose to go with exim4, I found it the easiest to understand. Which you can take anyway you like!

As I said I’m running a local domain, and one the way out the e-mail addresses are rewritten for the company domain, otherwise the reciever couldn’t reply. This I have setup using LDAP in the schema. I am using the mailRoutingAddress field. Perhaps not what it is intended for, but as I am the admin I can do what I like.
On the way out a e-mail has its from:, reply-to: etc.. fields re-written with this address. I haven’t bothered to rewrite on the way back in, leave that for another project.

Now my kids’s e-mail accounts are just for fun. They don’t have a routing address setup in their accounts. But they still sent e-mails out to people. And when that person received the e-mail, they could not reply as the appropriate fields had not been re-written. So I needed a way to stop the e-mails going out.

They way I looked at it in exim4 is that I needed to write a special router to perform a couple of checks and to fail and produce a message if a user’s mailRoutingAddress could not be found.

This is how I modified the


local_only:
   debug_print = "R: local only for $local_part@$domain"
   driver = redirect
   allow_fail
   domains = !+local_domains
   condition = ${if match_domain{$sender_address_domain}{+local_domains}{yes}{no}}
   data = "${lookup ldap {ldap://localhost/uid=$sender_address_local_part,\
     ou=People,dc=my,dc=domain?mailRoutingAddress}\
    {$local_part@$domain}\
    {:fail: Your account is not allowed to send mail outside of the local network}}"
   redirect_router = next_router
This router is only enabled where the outgoing domain is not one I am looking for. The condition like is perhaps optional here, but it keeps the router from failing if I use a purely virtual e-mail address (I think). It then performs a lookup into LDAP checking for the mailRoutingAddress. If it has a value then the router will redirect to another – a time saving measure as I don’t want to loop through this router again. Or it fails and produces a suitable message.
And that’s it.
Posted in domains, exim4 email, ldap | Leave a comment

Saving away outgoing emails in Exim

In our office we use internet fax by myfax.com who are simply excellent btw. Stay away from the other guys.

Incoming faxes are dropped into a common folder via a bit of procmail script in a common user’s home folder. This user has shared folder setup, see my other post on setting up shared folders in this way.

However, from time to time there is a need for staff to see a fax, which has been sent by another colleague. Usually one colleague had to forward the e-mail to the other so they could view it.

Today, I decided that was a small simple issue that could easily be resolved. We use Exim4 as our e-mail transport. Exim4 is excellent, but I find its documentation a little on the terse side. But I persevered and found two blog entries which helped me out immensely.

The first blog explains how to setup a filter file to truly blind copy all outgoing mail from a user to another user. However for Ubuntu users the file setup mentioned will not work. Ubuntu uses the flat-file configuration to make life simpler for user, and it does! So this blog  entry sorts it all out for us.

Now if you’d rather copy it to the shared folder, and your user’s have rights you can use the Exim save command documented here. I plopped it through as e-mail and let procmail take care to drop it into the correct box.

I hope you find this of great use to you and if you have other solutions, feel free to let me know by posting below.

Posted in email, Exim, exim4, filter, outgoing, Ubuntu | Leave a comment

Strange error on closing a .mdb

If you are getting a strange error about collation sequence in Access 2010 when closing a .mdb file,
Something like “Selected collating sequence not supported with the specified file format”.

When you look this up you will find advice to set the collating sequence to general. However this will not work. And in some cases the database will become corrupt.

The most likely cause is a recent Service Pack to Access, SP1. The latest Service Pak of Access 2010 appears to have a bug in the compact on close feature. The cure is to turn this off.

Posted in Access, collation, compact on close, error, service pack | Leave a comment

A begging letter

The web is a massive thing. You’ll find all sorts on here, good and bad, in relative measure of course. There are those what would consider blog on Access bad.

I’ve heard about inbound marketing. What it is exactly I am not sure. It sure does fit for a introverted person like myself, I can even speak with my parents for all that long!

But does it not all sound like inbound marketing is a new way of writing a work begging letter. It is a way to attract people who are looking for someone to find you. I thought if I blog on Access topics people will find me. So far it hasn’t worked out that way. Maybe they are coming here – though from stats they aren’t – and taking the knowledge.

Any inbound marketing tips and sites are welcme. I’ve used up my book budget this year so won’t be able to buy any of those.

Posted in Access, begging, hire me, inbound marketing, MS Access, unemployed, work | Leave a comment

Random crash of Backup server

I have been trying to nail down an issue with my backup server. I thought I had solved it with a boot option of noapic , because the server worked fine after this setup. It booted reliably up each time.

Until yesterday, when it did something that it had done before. That is the NIC seems to turn off all by itself and then the OS sort of hangs. Before this point the server had sent out its e-mails and even started a couple of backups. I’m not able to login at the console, so the only option I have is a power off and then a boot.

Checking through the logs reveals nothing. DMESG, syslog, messages reveals nothing. No panic, nothing. All I can see is that after sometime backuppc can no longer ping machines and then backuppc soon stops - perhaps because the server is now hung. Pinging the backup server does not work either, so the server really is locked up.

It is very annoying to say the least. The previous server was rock-solid in this regard. It was extremely slow, but at least it booted and stayed up. This maybe because it had a more modern BIOS than the current unit. Which makes me think I will now have to hunt down a updated BIOS.

This really is the first Ubuntu/Linux unreliability I have had in over four (4) years of using Linux.

If anyone has some place to begin, please don’t hesitate to comment. I am running Ubuntu 10.04 LTS server edition. Its only purpose is to run Backuppc and this server is woken by WOL each night to start the backup and then shutsdown early morning when all backups are done.

edit: Just to let you know that this appeared to be a hardware issue and I have switched everything over to the original backup machine. Which is much slower but at least works. I now need to wonder what the problem is.

Posted in APIC, crash, hang, kernel, logs, network, Ubuntu | Leave a comment