I’m switching directions on my quest to understand OverDrive Media Console. I do know a bit more about how OverDrive on Android works and there is much learning to be had in the ‘activation.xml’ and the sqlite3 ‘omcDB’ file.
I wrote a Ruby script to extract that Abode Digital Editions (ADE) key from activation.xml and it’s identical to the what the Python script gets from Windows and ADE. It’s just a key. I’ve got a valid account at the Library and with Google and Adobe. I’m not breaking any laws to get the key. Using the key to remove DRM, that might be an ethical issue but the python scripts are all over the net. Distributing the cracked epub would be illegal and there is no personal gain to be had for that. I did write a Shoes (ruby) GUI script to help automate the download of the files from Android device and extract the key. I ran afoul of Shoe’s well known problems with gems and Shoes (net-ssh, net-sftp for my script). I thought about the problem I wanted to solve. It’s a pure Linux issue — it’s not a problem to solve on Windows or OSX.
So, I switched to green_shoes where it uses the ruby/gems in the system (or rvm) instead of the built in, self contained gems of Red Shoes (rvm can really confuse you about gems and (original Red) Shoes.) I made a few minor changes to the hash syntax. Success. As in GUI success! I didn’t finish debugging the sftp code against the android device because I realized it’s kind of silly to download obscure files from Android that basically haven’t changed (the activationkey will never change) if you use the same public library
There is no OverDrive Media Console program for Linux or a Linux ADE so I can’t download the epub that I checked out of the library from Linux. ODMC on Android doesn’t tell me that I’ve downloaded the book before although it does know (I’ve seen the DB) Why waste a library checkout if I’ve already read the book, let someone else have that digital slot. Ebooks can only be borrowed so many times.
The nearest Linux equivalent to Adobe Digital Digital Editions is Calibre. There is a $2.99 Android app that will talk to a Calibre server (free) when configured. So, with the vast tools at hand and half baked knowledge why couldn’t I write script to accept and decode the .acsm download like ODMC or ADE do and then I could pass the .epub file(s) into Calibre.
One reason that the folks at Adobe and OverDrive don’t write Linux versions of their programs is that there is no standard GUI or language. Python, Ruby, GTK, GTK++, GTK2, TK, Nothing is standard. Sadly, they miss the point with their Purity blinders. I’m going to use Green_Shoes and Ruby and if I’m successful, it would only take a day or two of Python/TK work to convert. There are many reasons to use Python (TK?) instead of Ruby and Shoes (green shoes) for this acsm downloader attempt. Many very good reasons to use Python instead of Ruby.
But, I like Ruby more when I don’t know what I’m doing and if you need a small GUI, Shoes (green or red) is so much simpler. IMO. It could be I’ll end up using Python (did I mention may reasons I should).
So, I did a wireshark capture of eth0 traffic when I used (hold on for bumpy ride) a Virtual Box VM of Windows/Xp/SP2+ running Firefox connected to my OverDrive Library account’s when I download the acsm file and passed it off to ADE. Thankfully, that seems to have captured the VM traffic along with all the DLNA chatting and the router chatting. I haven’t dug into the wireshark log very deep. Just enough to know that there are http PUTS and some SSL key exchanges with multiples website that claim to be your Library. Stuff you won’t see in a browser. I also see some odd (to me) google safesite traffic. I don’t remember asking for that help. So many discoveries ahead.
[update 1]
Looking at just the HTTP traffic from the wireshark capture, it’s about what you would expect. A series of xml exchanges, then the epub+zip download and then a confirmation (XML). There appears to be lots of ‘echo’ data in the XML passed around as in you have to pass back what was sent plus some other stuff more appropriate to the phase of the transaction. I strongly suspect that different users agents (ADE/flash vs Android ODMC) will get you different different interactions. For certain the ODMC will get you a differently locked file.
I’m leaking enthusiasm quickly. The SSL exchanges goes to akamai which I suspect is to verify certificate/keys/stuff and a magic token is returned. I don’t want to decode and duplicate all that nonsense. On the other hand, maybe the cert check is just some thing Adobe did because it seemed like a good idea and it doesn’t really matter. It probably matters.
I am very interested to see your “Ruby script to extract that Abode Digital Editions (ADE) key from activation.xml” As someone with access to Linux and Android only, it makes this whole process very difficult. As far as I am aware, there is nothing illegal about extracting my Adobe key. This is some great work. Please let me know if you’re willing to share this code! I certainly share your frustration at Adobe and OMC not having any Linux solution available. Thanks!
I didn’t pursue the ruby script because it would take a lot of time to decode and replicated the ODMC protocol with no guarantee that it can be done (SSL/Cert issues). I download the book in Windows (in a virtual machine), ftp it to the Linux system, Load it into Calibre (it has a user installable plug in to remove the DRM). Calibre also has an book sharing server which I wanted. On the android device, I use Aldiko as a book reader because it knows how to talk to the Calibre server.
I realize this is not exactly what you’re working on, but I notice that OverDrive distributes DRM-free MP3s — but to get them you have to use ODMC on a .odm file. I haven’t installed ODMC (though ADE works fine under wine to get me .epubs, by the way) and I (foolishly?) hoped that it would be as easy as pulling the filenames out of the (XML) .odm file and firing off a wget. Apparently not. The three-part overdrive extractor for iphones claims that their (somewhat different) URLs can simply be wgetted from, but all I get are 403 Access Denieds.
Have you tried wiresharking MP3 audiobook downloads?
@Anne
I have tried something similar. I once checked out Hayeks “Road to Serfdom”. It had 5 mp3 files and downloadable urls inside the XML. You have to look deep in the XML to find them and, for me, the first minutes or so of speech had odd dropouts. Not saying is DRM, but you know? The real problem was I lost all the chapter info metadata so I couldn’t go to chapter 12 or to the last chapter I heard because MP3 players/clients (generally) don’t have that ability in the middle of a file — they could but that’s another can of worms to open.
Back when my device was a bog standard Color Nook using the Overdrive Media Player (not the Android its flashed to now), it would pause at odd moments while it downloaded the next MP3 file. There are more than enough hints in the XML that what I saw was indeed what was happening. Overdrive AudioBooks have expiration dates but that is enforced by the client checking the home base or it’s Android DB. If someone was so inclined, that would be easy enough to get around. Building a bundle that players could accept as expiration date free “AudioBook” ? That might take some work.
Thanks for mentioning the Wine option. I vaguely remember testing it and it did work with ADE. I also had to re-install Wine & ADE every time I updated Ubuntu versions. Easier for me fire up a cranky old XP SP2 vm. Wine is is viable option and it would remove the ftp step in my process.
Turns out OMC works under wine too, yielding simple .mp3 files in a directory you get to choose. I’m guessing it’ll delete them when my time is up. I was able to dig through the xml and find both a baseurl and a filename, but I couldn’t assemble them into a fetchable URL. I guess the thing to do is make OMC fetch a book through a proxy and record the traffic so I can see if they’re doing anything complicated.
It is quite complicated by design, in order to hide the details from people who would circumvent the DRM and return by dates. I’d use wireshark rather than a proxy server to capture the traffic (better filtering and more details that might matter). But your network needs to be quiet like a small home network and it’s a pain to learn. I believe there a public specs for what an Audiobook bundle should look like just as there are specs for EPUB. Whether ODMC follows the specs is unknown or disputed.