Thursday, January 31, 2008

Project Gutenberg for SONY Reader!

Alright, our office went on a 2-day ski trip to Whistler (one of the Google perks), and since I'm not an enthusiastic skier, I stayed behind to finish up some work, play some Halo, and do other fun projects.

One of my recent fun projects is a proxy of Project Gutenberg web site that adds ability to translate to SONY Reader. I was playing with it for a while now, and it slowly shaped itself into ~1kloc of Java code that actually seems to work.

So I thought I'd throw it here to see what happens. Note that this is ALPHA code. There is a good chance that I will take it down soon to fix bugs. It is also a standalone program running embedded HTTP server - eventually it will migrate into a servlet on Apache. But right now it seems to be doing something useful, so feel free to play with it.

Several notes:

  1. It is not intended to replace the official Gutenberg web site. Please go to http://www.gutenberg.org directly if you need to download stuff other than SONY books.

  2. This is running off my DLS connection. Please please please do not be a bandwidth hog. Absolutely no crawlers! Robots.txt is set accordingly, and there are abuse counters, so if you crawl, YOU WILL BE BANNED. FOREVER! Worse, crawling may trigger P. G. abuse logic, and then the web site will be down, because I won't be able to get to them.

  3. This is ALPHA. This means bugs, potentially, a lot of them. Please be a good member of community, and if you do find a bug, leave a note with repro steps, a URL that did not work, etc. in comments section to this article. Please take a look at previous comments, the problem may have already been reported.

  4. If you are a hacker - you can probably hack this machine. But its image gets wiped every so often, and there is absolutely nothing else on that PC. And it's physically outside my home network, on its own IP. So your triumph will be hollow and short-lived.



Alright, without further ado, here's the link:

http://www.solyanik.org:9000

Give it a whirl!

UPDATE 02/03/08

I've fixed a ton of bugs in it today - I am actually feeling quite good about the code now. So I am upgrading it to ALPHA! I could use some feedback from users though - so if you did use it, please tell me how it went!

Known issue: makelrf which I use to translate the texts crashes on some files. When this happens, you get a 403 page.

TODO:

  1. Move to Apache running as a servlet from current embedded web server.

  2. Move to a newer translation technology from makelrf.



When this is done, I will call it a BETA.

UPDATE 02/05/08

I've found and fixed the problem with crashing makelrf. It had static buffers for title, author, and description, which was not so big - 40-something characters for title, for example. It was frequently overflowing and crashing.

I've made my software limit the strings to the sizes that makelrf supports. If you tried to download a Reader book from the site but failed, try again. It should now work. In my own testing, the site has been quite reliable recently.

16 comments:

Anonymous said...

WOW ! That is so cool ! I was waiting when you will do that. Not that I own Sony ( or planning too), but this is very nice project. I really like how fast you started to "ship" staff :-) after you left big M ! Keep it up!

Sergey Solyanik said...

Well, the more people in the pipeline between the dev and the customer, the longer it takes to ship, right? And Microsoft was adding non-dev staff in droves recently...

Anyway, this was supposed to be a one-day project, for someone who knows anything about web servers and HTTP protocols. Since I knew nothing at all, I had to learn, plus carving a day worth of work from my nights was hard :-).

Xpectro said...

Sergey.

No words to thanks you.

I live in Colombia and I have a reader... but I was stuck with the ugly format the PGutemberg books came into my device.

Now...

Thanks for making this Xmas night again.

Anonymous said...

This is superb!

I used it for a few books, but didn't want to snag too much of your bandwidth.

I'm going to see if I can make a local proxy based on your work, so I can not over access your bandwidth.

It is striking to me that Sony offers no tools for converting documents with the Reader.

Sergey Solyanik said...

There's enough bandwidth for any human use (and all humans are welcome to use it!), I just don't want it to be crawled - as in running software that would attempt to download the whole site - because that would mean downloading gigabytes of data through this connection, and that will bring the site (and my internet access) down.

Anonymous said...

Had to write and say thank you. What a great program this is. I really appreciate the work you put into this project. I have downloaded a few books and i am reading my second downloaded from your site. They work flawlessly, perfect!

MBK said...

Thank you. I live in Australia and was given a Sony eBook by my son. Sony will not sell books outside of the USA so I did not know how to get any books in the eBook format. Thank you again, you have made an old man happy!

Unknown said...
This comment has been removed by the author.
Unknown said...

Is the source code for your web app for sale?

http://1-800-magic.blogspot.com/2008/01/gutenberg-for-sony-pre-alpha.html

Anonymous said...

THIS IS GREAT.. THANKS SO MUCH !!!
Maybe once you have an Final version you get get this directly into gutenberg!

Thanks again... many more books I wanted, but can't read on my sony reader.

You are awesome!

Anonymous said...

OMG. I have been looking for DAYS for something like this since I got my Sony ereader. You totally, totally rock.

Anonymous said...

Thank you- very neat. It opened easily in eBook Library. I am still new to this and am irritated by the eBook Library in how I can't drag it from that view into the library of titles but have to go and find it in the obscure temp file.
From a non coder since I did Fortran thanks again. Regards Alastair

Unknown said...

I can't believe that you have done it-congratulations. I have just downloaded without any problem whatsoever.Thank you.

Anonymous said...

This works really well for me too...

Just downloaded a book dropped it on the SD card in my Sony Reader. Amazing!

I am an educator - and making stuff like the Gutenburg collection widely usable in alternative formats is great for me.

I too struggled with nasty formatting before with .txt files, etc. - so I never bothered to do much with the device with kids much.

Unknown said...

The Sony reader is a great device especially for those who travel a lot. However their site is really hopeless. I had so many problems and looked on the web to see if I was alone. Not so. There were many people who could not get their Reader to connect with the Sony programme and had received little or no help from Sony.Then I stumbled across this programme quite by accident. It is well designed and what is more it works! If the Reader does not take off it will be the Sony Web site that contributes to failure.If Sony was smart they would buy this programme immediately or better still hire the creator who deserves rewarding. Well done again.

Anonymous said...

Thank you so much! A friend and I are in India for four months and brought our Sony e-books, but we were running out of free books that looked interesting from the Sony Classics. I just downloaded a book without any trouble...amazing! Now there are more books avalible than we could ever dream of reading--I'm so happy right now!