Larry’s Personal & Tech ramblings

I purchased a 30GB iPod a while ago because we thought we would be doing game development on it. The plans fell through, but instead of letting it collect dust in a drawer, I decided to let my family use it. It’s a well designed device in many ways, but the Windows software is truly lacking. I refuse to purchase low quality DRM’d music from the iTunes store, so all of my music resides on CDs. I’ve imported most of my CD’s into Windows Media Player and the WMA files take up about 11GB of hard drive space. I much prefer WMP-11 to iTunes to listen to my music. In order to listen to my music on the iPod, it needs to pass through the iTunes software. This is a much more unpleasant experience than it has to be. iTunes allows me to import a “folder” of music at a time. When I add CDs to my collection this wastes considerable time because iTunes has to convert each WMA file into its preferred format. I have a dual core AMD 4800+, fast memory and a fast hard drive. From what I know about performance, the iTunes music converter runs at least 5X slower than it could if someone at Apple with some brains would optimize the code. I see this as one of the many ways that Steve Jobs punishes Windows users. Besides the possible loss of quality in doing another format conversion, the iTunes files WASTE ANOTHER 11GB OF SPACE! The only reason I have to suffer through a slow conversion process and have 2 copies of my music is because of the endless conflict between Apple and Microsoft.  I thought this was the worst of it until my hard drive containing the iTunes files died. I plugged in my iPod and asked it to “sync” thinking that the music on the iPod would be copied back on to my new, empty hard drive - WRONG! iTunes gleefully erased all of the music on my iPod and made me start all over again. I guess I’ve been spoiled by Microsoft’s use of the word “sync” which means that data goes in both directions.

Apple is usually good about thinking through usability and design in their products, but deleting all of my music without even asking me is a big error.  I’ll keep my CDs for now.

WordPress (the company which hosts this site) collects some interesting statistical data on the people who visit the blog.  To me, the most interesting data is a list of the search words which direct people to this site.  Since I started including the JPEG and ARM keywords in my posts, I’ve seen a steady stream of people searching for basically the same thing: Free optimized source code for decoding JPEG/MPEG images on ARM devices.  I’ve done such searches myself and have come to the conclusion that it’s not available.  For anyone who has done research and invested tons of time and energy into writing optimized code, it is unlikely that they will be willing to give it away for free.  There are plenty of open-source and free projects on the internet that are valuable and professionally done, but there usually comes a point in a project’s lifetime when the author commercializes it to get compensated for the time invested.

I try to share my knowledge and experience with the developer community; I understand the frustration of wasting precious time locating resources or coming up with workarounds for problems outside of (or within) my code.  I also make a living writing software, and so I must write code which is worth compensation from my customers and maintain innovative solutions which compare well with my competition.  The geek in me would love to have an open discussion about the fastest way to decode Huffman encoded data or minimize the calculations in the IDCT, but as a consultant, that would be self-defeating.

The “trade secrets” are visible in the source code, but hidden in the object code, so licensing object code will incur less risk to me and therefore cost considerably less.  I’ve licensed my code to various companies for values ranging from several hundred dollars to tens of thousands.  The price varies according to the risk and time required.  Companies needing help with ARM optimization issues are encouraged to contact me.  The amount I charge for my time or code is usually far more economical than having other programmers spend time trying to invent what I’ve already  got working.

The two changes in 0.27 are the release of the Desktop PC version of SmartGear and the addition of the “PAUSE” button to the MasterSystem emulator (mapped as “Select”).

The product page for SmartGear is up –>SmartGear Web Site

You can now download friendly installers for Pocket PC and SmartPhone. The installers also install a Desktop PC version of SmartGear (never released before). A paypal button is at the bottom of the page to allow you to register.

I thought it would be useful to re-run the tests with the C version of my JPEG code. From the results it appears that memory bandwidth is the real limiting factor to the speed and the pixel colorspace conversion gets the most benefit from my optimized ARM assembly language. Also it appears that the OMAP gains more from optimized ASM than the XScale does. Here are the numbers:

C-Code:

PPC: thumbnail: 10.7 milliseconds, DC only: 968 milliseconds, full res: 3734 milliseconds.
SP: thumbnail: 25.1 milliseconds

Mixed C and ASM

PPC: thumbnail: 8.8 milliseconds, DC only: 830 milliseconds, full res: 2700 milliseconds.
SP: thumbnail: 15.1 milliseconds

The load times for the “DC only” and “full res” tests include the time taken to read 4.3MB of data from RAM through the WinCE file system.

These results make sense in that the real benefit of optimization comes from fixing the algorithms and reducing memory usage. The optimized ARM assembly code is certainly helpful in speeding things up, but won’t offer an order of magnitude improvement over what the compiler generates.

The great thing about the ARM architecture is that the more I look at a piece of code, the more ways I find to optimize it. The conditional execution, barrel shifter and optional setting of the processor flags create many opportunities for optimization. I’ve spent some more time optimizing my ARM asm JPEG code and now have some hard numbers to publish. I used a HP iPAQ h2210 Pocket PC (400Mhz PXA255) and a HTC Hurricane SmartPhone (195Mhz OMAP 850) to do the testing. I was able to load the file from RAM on the Pocket PC (to reduce file I/O delays), but not on the SmartPhone. The SmartPhone file system does not use RAM for file storage. The slow speed of reading from the miniSD card overtakes the amount of processing time in the tests, so the only test that was run on the SmartPhone was decompressing a 160×120 thumbnail image in RAM. All tests were to decompress the image to a RGB565 bitmap. The thumbnail test decompresses the 160×120 EXIF thumbnail image. The “DC only” test creates a single pixel from each MCU (the 3072 x 2304 image is loaded as 384×288). The “Full res” test decompresses every pixel of the image.

PPC: thumbnail: 8.8 milliseconds, DC only: 830 milliseconds, full res: 2700 milliseconds.
SP: thumbnail: 15.1 milliseconds

The speed difference between the two devices is to be expected considering the different processor and memory bus speeds. The “DC only” test is useful because it shows the relative speed of Huffman decoding. The file size is 4.3MB, so in 830 milliseconds the code was able to decode all of the MCUs and produce a single pixel from each one.

I’ve uploaded the sample image to my web server here: CIMG2209.JPG

The image was taken with a Casio EX-Z750 and depicts a relatively complex scene with many fine details. Like most cameras, the Elixim series saves JPEG images with 2:1 horizontal color subsampling (when set to maximum quality). It’s not unreasonable for a point-and-shoot camera like the Z750 to save images at a less than optimal compression because the image coming off the CCD isn’t that great to begin with. What irks me is that cameras like the Canon 20D do the same thing. With a good SLR lens and imager, the Canon should allow you to save full res color JPEG images.

Comments?

A while ago AOL decided to charge separately for their dial-up service and allow free email accounts. This news doesn’t seem to have reached all AOL customers because I’m still finding people that are paying for AOL dial-up access thinking that it’s necessary to keep their email address. I understand their need to keep an AOL email address because they have business cards or advertisements printed with that address and can’t easily change it. The other night I asked a neighbor of mine if he was still paying AOL to keep his email address. He’s the third person I asked and the third person who is paying without the need. Our neighborhood has a contract with the cable company to get discounted cable TV, so most of us also have high speed cable internet service. If you’re paying AOL $15-$22 per month and paying for DSL or Cable internet, you can probably save yourself some money. Here’s AOL’s description of their free email accounts:

AOL Free email help page

If you’re feeling grateful that I just saved you $180 per year, feel free to send some of it to me

Let me start by saying that I’m not a big fan of Apple. Through the years I have always felt that they extracted a few too many dollars from their customers and left them with a few too few options. I’m glad Apple released the iPhone because it will stir up the mobile market a little and inject a few new ideas into the mix. The truth is that the iPhone is very much a 1.0 release - missing several important features and designed to be more of an improved iPod than a good phone.

1) As a music player it falls short of being usable; 8GB of memory is not enough to store even a fraction of most people’s music collections.

2) As a phone, using a touchscreen is a mistake. A phone shouldn’t require 2 hands to operate nor require your eyes to be focused on it to make a simple call. Get ready for iPhone-related car accidents.

3) Choosing AT&T was a big mistake if you’re touting your device as a great wireless web browser. EDGE network speeds can be OK, but they’re never great.

4) Most of today’s expensive smartphones come with a much more reasonable price tag when you’re roped into a high priced data plan for a minimum of 2 years. For the rate plans that the iPhone offers, it should be free with the 2-year contract.

5) Apple fanboys will drink the kool-aid as always and flaunt their wondertoy.

Even though HTC won’t admit they released the “Touch” as a response to the iPhone, I think we can see through the smoke. The Touch is a nice little Pocket PC, but it has the same issues as the iPhone; in other words, I would never carry one around as my main phone.

For me, the HTC S710 (Vox) is the epitome of SmartPhone design. It’s small, light, tough (no touchscreen) and has a convenient slide out keyboard. This is the phone I’ve been waiting for since Microsoft released the “SmartPhone” platform.

Let the flaming begin…

I have always typed better on the old “clicky” IBM/Lexmark keyboards. There is probably some combination of tactile and auditory feedback that helps me type my fastest. I bought a bunch of broken ones from a liquidator around 1993 and managed to put several complete, working ones together. I’ve tried to keep them functioning and moving them forward with me as I’ve upgraded my work computer (and luckily it still has a PS/2 port). Along the way I’ve always wondered how long they would last (the switches and the controller). Well, last week I got a partial answer. My computer booted up with the dreaded “keyboard error” message. I figured that the cable had worked its way loose, but that wasn’t the case. Apparently the controller board had died. It’s not worth spending the time to determine if it was the 6805 or some discrete component. Impressive is not quite a strong enough word to describe its service record. In today’s disposable society, it’s simply astounding that a piece of consumer electronic equipment could work continuously for 8-16 hours a day for 14 years. Luckily I had a spare controller board and brought it back to life for a few more years

An odd title considering that JPEG is a cryptic image compression standard.  My idea of fun is optimizing code until there’s nothing left to improve.  I decided a few weeks ago to take the plunge and rewrite the 3 core JPEG decode routines to speed up my imaging code.  One reason was that the great majority of cell phones today are based around the TI OMAP architecture typically running at around 200Mhz.  These devices seem slow at working with images, so I thought I could help that situation by speeding things up to improve both battery usage and the user experience.

The important, “inner loop” routines of JPEG image decoding are the Huffman decoding of the MCU (minimum coded unit), the IDCT (inverse discrete cosine transform), and the output stage (turning the YCrCb pixels into RGB pixels).  All 3 routines together turned out to be only a couple hundred lines of ARM code, but the result of rewriting it from C was quite dramatic.  The original C code has been optimized and tested over a long period of time and was in good shape to begin with, but C isn’t so great at bit manipulation and squeezing the most use out of register variables.  It took several iterations to get down to the bare minimum of code, but I’m quite happy with the results.  I used ARMV5 instructions, but made sure that the code performs well on both OMAP and XScale CPUs (unlike Intel’s integrated performance primitives).  Luckily my previous performance testing of the multiply instructions helped guide me to save a few clock cycles off of several routines.  The purpose of this work is threefold:
1) I’m readying a new version of my imaging application (PQV - Pocket QuickView) for Windows Mobile and need it to be competitive with other products.  I pride myself on having the fastest viewer available.
2) I have been staring at the C code for a long time and wondering how much better it could perform if written in optimized ARM asm.
3)  I believe this code has value to anyone doing imaging or video on ARM based devices.  Web browsers, image viewers, camera applications, video players can all benefit from this code.

I’ve been searching for the past week or so for customers of this code, but the typical response is the “not invented here” attitude standing in the way of improving products.

I will post some sample images and benchmarks shortly to back up my claims of fast JPEG decoding.

Anyone interested in licensing object or source code should contact me directly (bitbank@pobox.com).

I’ve had a vested interest in Windows Mobile devices since the very beginning because of my many years of Desktop Windows programming experience. It’s definitely a big advantage for Microsoft to allow developers to leverage their programming experience from the desktop onto mobile devices. Microsoft’s design philosophy seems to have been to bring as much of the desktop operating system as possible onto battery powered devices and put a new face on it which fits the PDA/Phone paradigm. The obvious advantage to this is being able to port code easily between the desktop and mobile devices. What I’ve been discovering more lately is that the one fatal flaw in this design is a non-issue for many (like me) and a deal breaker for others.

The problem I’m talking about is application and memory management on Windows Mobile. The current scheme allows the users to run as many simultaneous applications as they like. This is usually a good thing and allows mobile devices to multi-task just like the desktop version of Windows. The bad thing is that Microsoft discourages the “close button” and “exit option” from mobile applications. Instead they claim that the system will close applications automatically when resources get low. This is the fatal flaw.

Let’s suppose that your application is about to run on a phone that’s got a bunch of other applications running. Memory is running low, but there’s enough memory to run the EXE, but not enough to allocate space for the work that the app wants to do. The application is allowed to run and then gets a failing error code when it tries to allocate memory. The automatic memory management doesn’t appear to kick in because the app did succeed in getting loaded. The application tells the user that there’s not enough memory to do the work, so the user must go through many manual steps to shut down other programs to free up some memory.

This scenario probably occurs frequently with certain users of Windows Mobile products. Those users who run various software applications and don’t shut down their phones at the end of each day. As the days pass and they run different programs, the phone gets more and more cluttered with running programs and will get slower and more problematic as resources are used up. A friend of mine clued me into this situation recently and it dawned on me that this is probably a big issue for many users. My experience with Windows Mobile devices is that the operating system and applications work excellently and I am thoroughly satisfied with my phone/pda devices (I shut them down every day). I could see how many people would criticize Windows running on phones for the reason stated above. What’s the solution? My first idea would be to encourage application developers to include a real “Close” and “Exit” option on their mobile applications. The second idea would be to educate people to shut down their devices at the end of each day when they set them to charge. I don’t think there can really be any good automatic process inside of Windows which fixes this situation. Perhaps a sentinel application which pops up when resources are getting low and gives the user the choice of shutting down some applications. Anyone have an idea?