Compress Query Strings in JavaScript
Since query strings are limited depending on the browser, I wrote this function to grab redundant data variables and consolidate them into comma-separated values. For instance:
JavaScript:
foo=1&foo=2&foo=3&blah=a&blah=b
would turn into...
JavaScript:
foo=1,2,3&blah=a,b
This saves some room in the query string. Of course you need to write a trivial line to parse it on the server side, but this saves you some room for more variables in a query string. Here is my attempt at a JavaScript function that does this: (don't use this, use John's revised function below)
JavaScript:
function compress(query) { s = {type:[],value:[]} $.each(data.split('&'), function(n){ parts = this.split('='); s.type[n] = parts[0]; s.value[n] = parts[1]; }); var csv = ''; while(s.type.length > 0) { value = s.value.shift(); type = s.type.shift(); while((pos = $.inArray(type, s.type)) > -1) { value += ',' + s.value.shift(); s.type.shift(); } csv += type + '=' + value + '&'; } return csv.substring(0, csv.length-1); }
Sorry, forgot to mention that $.each and $.inArray are jQuery functions.
I messaged John Resig, and he wrote the function that was much smaller, and faster.
JavaScript:
function compress(data){ var q = {}, ret = ""; data.replace(/([^=&]+)=([^&]*)/g, function(m, key, value){ q[key] = (q[key] ? q[key] + "," : "") + value; }); for ( var key in q ) ret = (ret ? ret + "&" : "") + key + "=" + q[key]; return ret; }
UPDATE: John Resig followed up with a post of his own, titled search and don't replace.
How John's Script Works
You may be asking, but how does this work? - let me explain. First, the replace function accepts a regular expression as the first parameter, and the second parameter can be a string or a function.
JavaScript:
replace(/([^=&]+)=([^&]*)/g, function(m, key, value){
As far as I'm concerned, that lines is the magic of the script so I will go furthur.
JavaScript:
/([^=&]+)=([^&]*)/g
This is the regular expression that matches data that starts with ampersand and goes until the end or until the next ampersand. This extracts the data. The /g part loops through each match instead of just one, and does a replace on it.
The function he is passing in has the three parameters set by the regexp. "m" is the match, "key" is the unique identifier, and "value" is the entire string that the regexp is running against.
Next he sets the object based on the key, if it exists already than add a comma between after the current value followed by the next value.
JavaScript:
q[key] = (q[key] ? q[key] + "," : "") + value;
The last part of the code is simply looping through the, "q" object and outputting our new compressed query string. Brilliant.
Comments
"viewIDs=6653,7258,14600,15104,6521,1436,2005,2006,2007,2008,2009"
The first thing i tried was to convert each number after the initial ID into a relative measure of distance from the first so you could get:
"viewIDs=6653,605,-7342,504,8583,-5085,569,1,1,1,1"
Then you can use javascript base36 encoding Number("605".toString(36)) and die happy or take advantage of the fact that javascript base36 encoding is all-lowercase and further compress the redundant 1,1,1,1 sections using the remaining 26 uppercase characters--ie: representing -1 and +1 with "Q" and "P" respectively and come up with a final string:
"viewIDs=6653,gt,5ny,e0,-6mf,-3x9,ft,PPPP"
here is a quick example where 'result' is an array of the original IDs
For example, I maked faster then twice code :
var compress2 = function (data,stuff,q) {
stuff = data.split(/[=&]/); q={};
for (var i=0,len=stuff.length;i<len;i+=2)<br /> q[stuff[i]]=(q[stuff[i]]?q[stuff[i]]+", ":"")+stuff[i+1];
stuff = "";
for (var key in q)
if (key) stuff=(stuff?stuff+"&":"")+key+"="+q[key];
return stuff;
}
I tried to re-write flickr "contact parser" early and was defeated. I used exactly technique by John, but simply "split then run array" it were faster.
Then I was come upon this post end benchmark your code. Unfortunately timing were as with flickr.