| Matlab is dead. Long live Python |
[Mar. 30th, 2009|09:04 pm] |
Over the past year and a half I've incorporated Matlab into my daily work. It's a tremendously useful tool for experimenting with image processing and linear algebra. It's got a nice array-slicing syntax, so the second column of a matrix, M is M(:,2), and every other row in reverse order is M(end:-2:1,:). It also has some very useful easy graphing commands, so if you have an array of values sampled at a corresponding array of x and y positions, you can say surf(x,y,A) and get a nice 3D surface plot. It also has other nice language features like the ability to return a list of results from a function and assign those results separately as in [V,D]=eig(A).
However, it's got some serious serious shortcomings. First, it's closed source and expensive, so if you prototype with it, you have to then either port it to another language or kludge together scripts to call Matlab—not cool. Second, unlike any language other than assembly, values returned from functions are magic in that you can't index into them. So while the function eig(A) returns a vector of eigenvalues of A, if you want just the first one, you can do v = eig(A); v = v(1) but you can't do v=eig(A)(1). This is particularly tiresome with a language geared for matrices in which you might want to nest a series of functions indexing each as you go. That is, you couldn't do foo(bar(baz(A)(1:2:end))(:,:,0)), you'd have to assign each result to a temporary. Third, Matlab indexes from 1. Yes that's what everyone expects when they start programming, yes mathematicians index from one, but for twiddling with arrays, zero is just easier.
Finally, Matlab doesn't lend itself to creating functions or data structures. In general each function is in exactly one file, leading to terrible code reuse. You can add more functions to the end of a file but those can only (?) be used as subroutines within that file, leading to a terrible lack of code reuse. Worse, what little support Matlab has for classes is done through a special directory for the class and a separate file for each method(!)
Fundamentally, there is a need for an interpreted language capable of being run interactively but that is also a real standard programming language, not some kludgy interpreted dialect of Fortran. That something needs to have mathematical and graphing batteries included .
Fortunately, there is a collection of Python packages that have a very solid chunk of this covered: matplotlib, scipy, and ipython. Not only do they have all of the 2D graphing that Matlab has with a similar syntax, but they use the (awesome) Anti-Grain Geometry library for graphing, so you get plots way better than the aliased 1990s-style plots that Matlab dumps out.
Python has language support for array slicing and for returning tuples, but it is also a Real Programming Language in that you can write a linked list or a graph without having to hack it on top of arrays. It isn't designed with array programming in mind the way Matlab was, so the syntax for arrays is slightly more cluttered, but not by much. The big differences being that Matlab has special element-wise operators and has concise array construction notation, so a.*b in Matlab returns the array formed by multiplying a(i) by b(i).
So give it a try. Install the appropriate packages and start doing your scientific computing in an actual programming language.
ben@endash:~$ ipython -pylab
...
In [1]: x=r_[0:100:0.1] # Generate [0, 0.1, 0.2, ..., 99.9]
In [2]: plot(x, sin(x)/x) # Nice pretty plot.
Out[2]: [<matplotlib.lines.line2d object="object"at="at" 0xa0ee8cc="0xa0ee8cc">]
In [3]:
Can Matlab rasterize images to look this nice?:
PS
If you want a blast from the past, Python even has a turtle library, giving it the same drawing sematics as Logo.
from turtle import Turtle
t = Turtle()
for in in (0,1,2,3):
t.forward(100)
t.right(90)
Not only is Python making programming as fun as it was in my hard-core Logo days, it's doing it in a language that can simultaneously be used for Logo-like simplicity and Matlab and driving high-performance computing. Go Python! |
|
|
| On Fortran linear algebra "codes" |
[Feb. 12th, 2009|11:25 pm] |
There are a lot of solid, useful libraries out there for scientific computing such as ARPACK, CHOLMOD, LAPACK, and others. These all seem old (with some pages last modified over a decade ago) and all seem to work reliably, but all have lousy APIs with cryptic documentation lacking any examples; none of these Just Work.
For example, using ARPACK to compute the n smallest eigenvalue–eigenvector pairs of a large sparse matrix requires calls to two functions from ARPACK as well as a call to your own linear solver. (LU decomposition so the repeated x = A\b solves are efficient is left as an exercise.) First, you call DSAUPD in a loop.
What is DSAUPD? I think it's something like Double-Precision Symmetric Arnoldi U? Positive Definite. This function takes sixteen (24) arguments, all by non-const pointer. Some of these are magic numbers (as opposed to enums), and some of these are magic strings(!) I've seen this in other Fortran APIs; what are people thinking passing strings around? Sure for solving large-scale eigenproblems, the overhead is small, but it's just the Wrong Way.
It returns a magic number and you the calling code are supposed to perform different operations depending on what is returned until it says it is done.
As if that's not enough, you then have to call DSEUPD with 22 arguments(!!!); that actually gives you the resulting eigenvalue–eigenvector pairs you were looking for.
It's frusterating that this good code is out there tangled behind a lousy, archaic API. Boost.uBLAS provides a decent general-purpose generic C++ classes for linear algebra, but for some reason, Boost (which usually has libraries that Just Work) appears to provide no built-in solvers or wrappers to these old Fortran solvers. Am I missing something or are we really missing a good API to these tremendously useful functions?
The bright light in the midst of all of this appears to be SciPy which provides Python bindings to a lot of this stuff, as well as providing Python bindings for some nice nonlinear optimization libraries (which again wind up with kludgy Fortran calls at their core, but hide those gory details). Still, it seems like (short of rewriting these libraries in a more expressive language than Fortran) these things should be wrapped in a clean, efficient C++ interface and then wrapped in a nice friendly interpreted language like Python (or an ugly kludgy interpreted language like Matlab).
PS
Why must Fortran people call everything a "code"? 0xDEADBEEF is a code; ROT13 is a code (although I guess pedants would beg to differ); I'd even call MOV and NOP codes. A source file contains code and a library is a library, not a code. Come on people, talk like software engineers. </rant> |
|
|
| DTV |
[Feb. 11th, 2009|10:41 pm] |
We don't have cable; we watch a few shows on the Providence, RI CBS affiliate (heading: 212°, distance: 11.8 miles). With analog, we get OK signal. It's sometimes a bit snowy and occasionally black and white, but it's there and there is always good audio.
A month or two ago, the broadcasters all teamed up to put a DTV infomercial in the middle of Criminal Minds on all channels. We had been planning on getting a converter box and turning it on by February 17, but noooo, they had to go and make it seem like we couldn't trust analog TV to actually carry the last quarter of our crime dramas, making it worse than useless.
After filing for a $40 government discount (why are my tax dollars paying to help people watch TV?) I waited. And waited. And finally caved and went to Best Buy where (as usual) I got overcharged by $20, but walked out the proud owner of a DTV tuner.
I plugged it in and (after realising that the battery to the remote was wrapped in clear shrink wrap making it not provide any power) it worked crystal clear color picture and great sound!
Except for when it doesn't. When the signal fades, the entire screen crumples out and worse, the audio seems to go first. It might be watchable if the video dropped out first, but with the audio jumping in and out, it's worthless.
I just got a Philips MANT-410 Amplified Indoor dipole antenna, but it seems to be next to useless.
The infomercial said something about never needing rabbit ears; as it is we are this close: ⇥⇤ to giving up on broadcast TV and trying to just watch TV online.
All that said, antennaweb.org mysteriously lists a bunch of "Live Dates", including WPRI-DT channel 12.1, live 17 February. It thinks that will come in better than the otherwise identical entry without that live date. This suggests to me that they may be going to boost their signal strength on the 17th, but the station hasn't mentioned anything of the sort. I'll be really pissed off if they are making me jump through these hoops only to discover that I shouldn't actually expect DTV to work properly until the 17th.
For reference, I posted to this WPRI discussion board as well.
PS
How do antennas work? It's magic. Why is a tiny little spindle of metal the best way to pick up radiation from a long way away. One pair of bunny ears doesn't exactly subtend much of the sphere around the tower 11.8 miles away.
Tune in tomorrow for a rant about old-fashioned Fortran-based large-scale sparse eigensolvers that turn their inner loop inside out, making you do all the dirty work. Grrr. |
|
|
| IT Fun |
[Sep. 4th, 2006|10:16 pm] |
The other day we attempted to set ![[info]](http://l-stat.livejournal.com/img/userinfo.gif) mew0422 up with a wireless connection from her eMac in her third-floor room to the cable modem on the second floor. One requirement was that it needed to be done this weekend, so at that point online shopping was out of the question. The obvious solution was to buy a wireless router and a wireless USB dongle for her computer. According to the Apple guy at Comp USA, however, they didn't have any wireless adapters with OS X support. Since time was of the essence, we opted to go for a Linksys “Wireless Game Adapter”, essentially a wireless–Ethernet bridge. (Comp USA had a Linksys Wireless-G Game Adapter for $100; Best Buy had a Linksys Wireless A/G Game Adapter for $50. Score one for Best Buy.) We started with a Hawking Technology Hi-Speed Wireless-G Router for $30 with a rebate to bring it down to $15.
Like many network devices today, both include their own web configuration, so you point your web browser at 192.168.1.x and can configure the device. Unfortunately, the directions are very short on details regarding exactly how to set things up. I don't know that much about networking, but I have written a simple ARP client so I'm not a complete newbie; this shouldn't be too hard.
For reference, here are some problems I stumbled upon:
- The Game Adapter does not provide DHCP (i.e., it doesn't tell your computer what IP address it should have) so if you just plug your computer into it and then point your computer at 192.168.1.250 like the instructions say, it won't work. The Linksys instructions say something like “plug the Game Adapter into your computer or into your router; it is recommended you plug it into your router.” What this means is “If you plug it directly into your computer, your computer won't get DHCP. Your computer may decide to invent its own IP address but then wont be able to route to your Game Adapter. In other words, either set a static IP on your computer or plug things in through a router so you get DHCP from that.”
- By default, the Game Adapter will clone the MAC address of the device plugged into its Ethernet port. The instructions say what this means, but not why. The reason you'd want to do this is that some cable modems only like talking to a single registered MAC address; by cloning the MAC address of your computer, you don't get stuck by this cable modem “feature”. However, as far as I could tell, this meant that if this feature was enabled, once the Game Adapter got connected wirelessly you could no longer use the Game Adapter's configuration utility because you couldn't send packets to the device because it had become transparent (a tube, if you will). This behavior made it look like it ceased to function as soon as you clicked connect.
- For some reason still unknown to me, the Game Adaptor would not connect to the Hawking Technologies router. I kept following their directions, clicking “Connect” and having the page reload with a status of “Disconnected”. The only feedback was on the router. There, when I turned logging on, I got the message A wireless client is associated followed immediately by A STA is expired. Google failed me on this one, as did Hawking tech support. You'd think tech support would at least have a complete explanation of all their error messages. I still have no idea what an STA is.
Interestingly, at least on a Sunday night both companies had responsive tech support, with less than a minute listening to music. My first Linksys call was even useful, explaining that I needed to get DNS involved somehow.
This morning, I gave up on Hawking Technologies after not hearing back from an email to level-two tech support about A STA is expired. We returned that router to Comp USA and grabbed a Linksys Dual-Band Wireless A+G Broadband Router (again Best Buy beat Comp USA, this time by $20 for a $50 product), with the logic that then Linksys tech support couldn't give us the “call Hawking” runaround. (A runaround I don't disagree with; I wouldn't want to be the guy at the other end trying to troubleshoot with a manual for only one of the two products involved.)
With the new Linksys router, there was still confusion. There were times when I could not load 192.168.1.1 (the location of the router's configuration page) even when I had a terminal actively pinging that address. Eventually I was able to get that page to load long enough to change the default password, enable WEP and set up the Game Adapter. After that, it seems to work perfectly.
In short what worked was: Plug computer and Game Adapter into Router via Ethernet (so the computer gets an IP address from the router). Configure Game Adapter not to clone MAC address, set passwords to something other than Linksys's default “admin”. Enable WEP on the router. Enable WEP on the Game Adapter and have it connect to the router. Plug the Router into the cable modem, plug the Game Adapter into the computer. In theory it all works. If it doesn't, set a static IP on the computer so you can go to the Game Adapter's configuration page.
The most exasperating aspect of this setup process was an inability to determine the state of the network components. It would be nice if components could explain their state a bit more clearly than via a web form you can't always get to. Perhaps an LCD on their front or a separate Ethernet port which would Always Work when you plugged a computer into it. It may cost a few bucks, but tech support, even outsourced to India, can't be free.
Questions
Why does OS X think it should invent an IP address for itself when DHCP fails?
What is the OS X equivalent of entering a 128-bit ASCII WEP password? On mew0422's house-mate's computer we were unable to enter a key longer than five characters. Apple says “ASCII - enclose the characters in double quotes ("). For example: "magic"” but that didn't work. The OK button still went insensitive for anything other than five characters (including the opening quote mark). |
|
|
| Tuesday the 11th |
[Jul. 27th, 2006|01:47 am] |
As mew0422 recounted yesterday, we got up early on Tuesday morning and headed up out of the foothills to Moro Rock (think Half Dome light and with stairs). The view was spectacular.
It's pictures like these that are why I got a grad ND filter.Without it, portions of these pictures would be overexposed.
Near there we drove through a felled tree. (I had heard of this; what I pictured was driving through the base of a standing tree. This isn't that but such a tree does exist, the privately owned Chandelier Tree well north of San Francisco.)
We checked out General Sherman, widely considered the largest organism. It is very big:
More trees, more chances to mess with various filters. Unfortunately there's no automatic way to know what filters I used; I'd need to keep notes. Hopefully in the future RFID will solve this problem.
Photo: mew0422
A young giant sequoia in front of a full-grown one.:
Next we went to Kings Canyon National Park, contiguous to Sequoia. Not being used to our crappy automatic-transmission car with lousy gas millage, we had to gas up in the middle of Kings Canyon. They claimed to be the oldest operating gas pumps in the country. It was cool to watch the gas level in the glass cylinder drop as it emptied into our tank. It was less cool given that the minimum purchase was 6 gallons for $25.50 ($4.25/gallon for 87-octane).
After that we drove down to Fresno where we spent the night. Now that I've spent a night there, I really have to wonder why somebody thought Fresno's was a good name for a restaurant chain in New York and Pennsylvania. I suppose it is a consolation that their food is about as awesome as Fresno is an awesome a city, so at least there's some truth in advertising. I do wonder why they have an Arizona/New Mexico/Utah theme when Fresno is no where near that, but I digress. |
|
|
| (no subject) |
[Jun. 17th, 2006|07:38 pm] |
Some people have complained about scratched iPods. I don't really care, but I've heard a lot of questionable rhetoric on this topic yet have seen nothing quantitative about the actual materials involved. One would assume that Apple engineers made a reasonable engineering decision in choosing the material (although I'm sure some would say the goal was planned obsolescence).
Apparently Apple claims “It's made of the hardest polycarbonate... You keep it in a pocket with your keys?” Well, I keep this cell phone in my pocket with my keys, change, and knife, and I have for three years. Its painted ABS case is scratched to hell, but the screen is almost perfect.
In contrast, I've been very nice to this iPod and have only owned it since February (when I got a replacement under warranty).
From what I can tell, the clear plastic in an iPod is Lexan, the same material as Nalgenes. Lexan may be tough, but it isn't very hard.
This raises two questions: What kind of clear plastic was used on my cell phone and why do I hear whining about scratches but no one saying Apple should switch to this other, harder material? |
|
|
| Radio! |
[May. 24th, 2006|11:28 am] |
As I mentioned in a previous post, my car radio recently gave up the ghost. I considered replacing it with an after-market unit, but I detest the idea of having a bling-bling radio with bright blue LEDs looking all out of place in my nice black dashboard with 1994 green backlights. If I wanted spinners, I'd have an Escelade; I have a Honda Accord.
With that in mind, last weekend I went to the Honda dealership. They claimed that, since my car is more than 10 years old, and a radio is not an essential component, they didn't stock my radio. They clearly have not tried to drive without a radio.
So today I went to my local junkyard and got a used radio for $35. Now I have music again (I just need to reassemble my dashboard). |
|
|
| (no subject) |
[May. 23rd, 2006|09:41 am] |
My car radio recently gave up the ghost. In pulling it out of the center console I managed to blow the fuse for my dashboard backlight. Driving at night without a speedometer backlight was annoying, but not dangerous and not problematic. It was frustrating, so before I left to drive from Greenfield, MA to Troy, NY (a two-hour country-road drive) I popped the fuse box to have a look. However, the owner's manual said nothing of dashboard lights and the one or two apparent possibilities proved incorrect. Not wanting to pull every fuse and not being able to see them well in situ, I gave up, closed the hood and drove.
I've posted pictures of this drive, Route 2 through the Berkshires, before; here are more. The graduated ND filter I've been excited about recently was invaluable with the high dynamic range you get in a hilly region at sunset on a cloudy day. These are all taken with a three-stop (23×) GND filter. I would like a harder-edged transition for sunsets like this; about a quarter of the frame is in the transition zone as it is. Unfortunately I've only found expensive hard-edged GND filters (~$100) as opposed to $20 for the ones I have.
Driving through Williamstown, I saw a cop on the side of the road. He pulled into traffic behind me, then turned on his flashers and stopped behind me when I pulled over. The officer helpfully told me that my tail lights were out and that he thought driving from there to Troy (~1 hour) wasn't a good idea. At his suggestion I popped the hood and started digging around in the fuse box. Again to my surprise, there was no fuse labeled “tail lights”. I started guessing and soon pulled a dead 15-amp fuse labeled “small lights”. I swapped it with the 15-amp fuse labeled “radio and lighter” and sure enough, I get tail lights and dashboard backlights. I got a written warning, thanked the officer, and was on my way.
So my question is this: Dear Honda, what idiot was reading a draft of the owner's manual, crossed out “tail lights & dash” and wrote “small lights”?
Update: Apparently I missed my school's 200th commencement this past weekend. But I've got to say, making the curriculum easy enough that 30 of 928 undergrad graduates could get 4.0s is unacceptable. (Fortunately they did just add grade modifiers this year and are slowly rolling them into lower-level classes.) |
|
|
| navigation |
| [ |
viewing |
| |
most recent entries |
] |
| |
|
|
