Tuesday, February 26, 2013

JPEG optimization tools benchmark

Intro

I have a web project that hosts more than 15.000 images of various sizes from 300kb up to 10MB.
In total these images take about 8GB of space. These images are served from a small VPS with 365MB of RAM and optimized LAMP stack with  Nginx as reverse proxy.
Since I am a performance freak, I want to speed up the image serving as much as possible.
But before tweaking any server settings I have to optimize the images themselves.
For this I started looking for linux JPEG optimization tools and found a few, no idea which one is the best so I decided to run a few benchmarks and share the results with the rest of you.

List of tools


jpegrescan- A lossless jpeg optimization script written in Perl. Created in 2008 and hosted for a long time on a pastebin page it doesn't inspire a lot of confidence, but why not.

jpegtran - A popular tool for lossless jpeg optimization.

JPGCrush - Found only as a comment on hacker news without any documentation or info. Lets do this!

jpegoptim - Maybe the most popular tool for lossless and lossy image optimization


imgopt - A rather simple bash script that uses multiple tools at the same time to achieve maximum optimization.



Benchmarks

For the benchmark I am going to use the following:
4 random images from /r/serverporn.
My own image collection of 872808kb.


First I am going to test the 4 random images
Name Type Parameters Total size before (KB) Total size after (KB) Difference
jpegrescan lossless none 3772 3556 -5.726%
jpegtran lossless -copy none -optimize 3772 3724 -1.273%
jpegtran lossless -copy none -optimize -progressive 3772 3560 -5.62%
JPGCrush lossless none 3772 3772 0% *
jpegoptim lossless --strip-all 3772 3724 -1.273%
jpegoptim lossly --strip-all --max=90 3772 3236 -14.21%
imgopt lossless none 3772 3724 -1.273%
*Not sure where is the problem. It run just fine with no results. I will be excluding it from the next benchmark.

And now some heavy testing. I have a directory with 2400 images and a total size of 853MB.
Lets see how the tools are doing on a larger scale.

Name Type Parameters Total size before (KB) Total size after (KB) Difference
jpegrescan lossless none 872808 756976 -13.271%
jpegtran lossless -copy none -optimize 872808 797764 -8.598
jpegtran lossless -copy none -optimize -progressive 872808 770764 -11.691%
jpegoptim lossless --strip-all 872808 819896 -6.062%
jpegoptim lossy --strip-all --max=90 872808 606260 -30.539%
imgopt lossless none 872808 832072 -4.667%


Summary

I didn't expect this, jpegrescan has the best lossless compression compared to all other tools including the most popular ones. Of course it took about 4-5 times longer to finish, but still, who knew.
And of course the winner is jpegoptim with lossy compression and quality of 90.
I think this the best option, since the users can not tell the difference between 100 and a 90 quality but the size reduction is significant.

P.S Do not forget to check out the new redesigned jsdelivr.com

Friday, February 15, 2013

jsDelivr 2.0 changelog and some thoughts

I am happy to announce jsDelivr's first redesign.
Its now based on Twitter's Bootstrap and features a number of improvements. 
Here is a changelog:
  • The origin server was moved to Github. It means that all changes made to the Github repo are synced to the CDN. I hope this will help to create a community that will help jsDelivr grow even bigger.
  • The homepage is not static any more. It shows the first 5 projects in the DB.
  • All results now only show 2 CDN links by default. A "Show more" button will allow you to see the rest of the links. The result is a much cleaner and browsable website.
  • Before, to view the links of an older version of a project you had to be on the project's page because the project name was taken from the URL hashbang. You could not use it directly in the search results. This was fixed and you can now use it anywhere you want.
  • When you request the update of a project you can now enter your email. This will resolve an issue I had. Half of the update requests were not processed because I could simply not find the new version. Now I will be able to contact the user to get more information. 
  • Github links were added along with the homepage. If a project doesn't have a homepage then the github link will be used in both cases.
  • Better UI and typography. Everything is bigger and cleaner.
  • Search improvements:
    • A small delay was added before an Ajax request will run.
    • Unique pages and permalinks. Before a "bootstrap" search request or permalink would match at least 3 projects containing the word "bootstrap". Now its considered a unique permalink and will only match the project named "Bootstrap". If you want to see the others you will have to type for example "bootstra" or "bootstrap."
  • Experimental use of tags next to each project. Currently the system can recognize some jquery plugins,wordpress, bootstrap plugins and fonts.
  • An "Integration" button was added. Each time its pressed, it will run an ajax request to get the file list of the project and version selected. After that it will output only the javascript and css files of the project, ready to be copy pasted into your website.
  • The back-end was partially rewritten, making the code more readable and easier to extend and use.
  • A proper FAQ page was created. Send me emails if you want me to add anything else.
  • A Network page was added showing the location of jsDelivr's CDN. Once it goes live more information will be listed with details of the setup.
  • packages.php was moved to http://api.jsdelivr.com/packages.php. Its now hosted by AppFog,Memcachier and Xeround. 
  

TODO:

  • Fill in the "About" page.
  • More information regarding jsDelivr's infrastructure and back-end.
  • Donation page.
  • Finally create a favicon.
  • Fix "Show more" when loading a different version. (no idea how)  
  • Get project info from Github via API. (worth it?)
  • A real API. (Any suggestions?)

Just stuff:

  • Lots of webmasters are skeptic about using jsDelivr because they don't know anything about what happens behind the curtains. So from now on I am trying to make it as transparent as possible. If you are one of those users please email me with the information you would like to know that would make you comfortable with using jsDelivr. I would appreciate that and it would help many people.
  • I am always looking for possible ways to promote jsDelivr. Let me know if you think of something. Plus if you want to guest post/be an author/interview/benchmark or something else, do the same thing, contact me.
  • I cant wait for the new infrastructure to be ready. Right now I just wait for the sponsor companies to release some new features that will allow me to use them with jsDelivr. HTTPS is my eternal problem, if it wasn't for it, right now the jsDelivr network would be huge.
  • Any developers out there that want to host something on jsDelivr? The rules are really flexible, just let me know. It would be great for jsDelivr to become your official CDN.
  • Same goes for WordPress developers. wp-slimstats already has integrated jsDelivr as a feature. Anybody else interested?
I think thats it. You can mail me at contact@jsdelivr.com or leave a comment here.
Sorry for any grammar mistakes.