The Computing Sapir-Whorf
Simon Pitt |
Monday 6th June
The eagled-eyed amongst you will have noticed that there hasn't been much happening on this site lately. This is due to a combination of reasons, many of them boring, some of them pathetic and one of them to do with flatulence (not ours, I hasten to add).
Since I last wrote I've got a new job in the BBC. Although it's not a technical job, I find myself writing more and more code these days. Generally because I choose to, but I'm getting to the stage where I can't imagine how I'd do my job without it.
I'm often amazed by the gaps people have in their IT knowledge. And I freely include myself in that category. Until a couple of months ago, I didn't know how to do a pivot table, and I consider myself quite an advanced user of Excel.
But it's often quite scary to see the ways people are wasting their time when just a little knowledge could save them hundreds of hours. Earlier today I went to help someone with Excel. He wanted to turn this into this:
He'd been on Excel courses and asked how to do it and no one had given him a solution. (If you're interested, the solution isn't that complex. Just add a column to the right and stick this in C2, and fill down:
To talk it through: check if the cell is blank. If it is, take the value from the cell above. If it isn't take the value that's in it.)
Sorry, I don't mean to start lecturing about Excel. I'm sure, you have better things to do. Like banging your head on the floor or something.
The scary thing is, for the past few months (if not years), he'd been facing this problem and had been manually typing the value in. On one occasion, he'd worked his way through a spreadsheet of 6,000 rows!
I remember in a previous job I'd once noticed a production manager typing figures into an excel spreadsheet, and then multiplying them in a calculator and typing the results into the next cell. I showed her how to write a formula to calculate them all. She nodded and said, "that's interesting, but I'm used to doing it this way so I'll carry on". (And I'm assuming here, this isn't because she has an intimate understanding of the Excel floating point bug)
I wonder how often I do things in stupid ways, simply because I don't know a way of scripting it, and I wonder how much of my time I waste.
(If you're reading this at home, and you're my mum, I hate to break it to you, but that bloke in The Social Network, Bill Gates and those guys that made Google (as you know them) are better at computers than me. I know you think I could make Google and Jesse Eisenberg reminds you slightly of me, but I'm afraid that's not the case.)
I've recently started to teach myself a bit more vbscript. I quite like vbscript, if only because when you have it on your screen it looks like proper code. I spend quite a lot of time writing HTML, which I never think looks very impressive. It's just standard words with <b> and </i> scattered around in it. And basically everyone knows it these days. Except my mum (see above), who still is impressed that I know to type and around a word and make it go thicker. Although to be honest, even she doesn't clap any more.
However, leave some good vbs on your screen and you can scare away all but the most hardcore computer users.
So I had a problem. And, luckily, I'd just been playing around with vbscript and it seemed to offer a solution. For reasons too stupid to go into I had to take a list of everyone who had an IT account, type it into a website, and see if they were listed or not. If they were, their information would be displayed, so I could pull it out and add it to a spreadsheet, if they weren't, I could move on to the next one.
It sounds fair enough. Slightly boring and repetitive, maybe, but do-able.
The problem is, there were 36,190 records.
Suddenly, it's not do-able at all. Let's say each one takes me five seconds to do, that would be fifty hours. Which when you factor in loo breaks, meals, talking to people about your weekend, would be about two weeks solid work. I think it's fair enough to say that this task is not possible. Even if you were to hire someone to do it, there would be so many errors you couldn't trust the result.
However, with vbscript, it's a relatively easy task download each webpage, test what's on it, and if what you're looking for is there, add it to a spreadsheet. In fact, the script I've written can do it in about four hours. The real bottleneck is the time it takes to download the webpage from the server, and there's not much I can do to speed that up.
The point is, knowing a bit of scripting not only saves time, but it can make things possible that were previously impossible. I suppose it's a bit like the Sapir-Whorf hypothesis (pretty much my favourite thing I learned at university - and I suspect I knew about it before I went): the words you know influence the way you think. In this case, knowing a bit of scripting changes the way you look at problems, and presents new solutions.
If you didn't know you could scrape webpages via a script you'd either have to find another way to get or estimate the data, or give up on it.
Sometimes it scares me to think of all the problems and inefficiencies that I can't solve because of my world-view. However, I like to think that every time I learn a new bit of code or a new tip for anything (this isn't just code, I suppose, it applies to everything), that's one less problem I'll face. I think as long as I don't go "that's interesting, but I'll carry on doing it the slower, less accurate way that I've always done", I'm getting less stupid. And I think that's all any of us can hope for.