Archis's Blog

April 17, 2014

“The password you entered does not match”

Filed under: Uncategorized — archisgore @ 12:00 am

A story of space…

When I was 10, it was already 35 years into the space age. Humankind had made it onto the moon. I was fascinated with reading as much as I could get my hands on, about the Apollo and Gemini missions. It would be my first introduction to redundancy. This was the era when computers were hilariously primitive. With no software development discipline, no formal verification methods, no historical learnings to depend upon, how would these machines be kept reliable? The answer was both elegant and ingenious – two separate contractors would build two independent computers in isolation, designed to perform the same calculations in real time. When they disagreed on their computations, that’s when the pilot would be alerted.

While it took me 20 years to understand what they did, they weren’t really providing redundancy in the true sense of the word. They were testing the system LIVE. One of the systems (picked arbitrarily) would be treated as the product, while the other system would be treated as the validation test.

It is a simple and powerful concept. When in doubt, simply rebuild the entire system all over and see if it matches. The chances of making mistakes in both systems is, without a doubt, very high. However, if it could be shown that the chances of making the same mistakes in both systems is very low, then you might be on to something. For very little knowledge of any software discipline at all, you could still build a pretty damn good quality system.

While you continue reading the rest of this post, try and answer the following question: What does a mismatch in calculations tell you? Try and enumerate the implications of that predicate. It’s going to be important later, because that will directly determine what we can do with that information. If we wish to do nothing about it, then having the validation in place, achieves nothing, but costs us a lot.

The story of ECC and Parity bits…

Anyone who’s taken basic CS classes knows about a parity bit. It goes something like this:

You know that game you play where you are standing in a row of people on stage.The game show host whispers a particularly tough-to-remember phrase to a person standing on one end. They have a limited amount of time to repeat it to the next person and so forth. Eventually the person standing at the other end has to repeat the phrase as they know it best.

Let us assume this phrase was “much would would a woodchuck chuck if a woodchuck could chuck wood“.

Now there are a few tricks each participant can use to ensure the person they relayed it to, got it correctly. One way I just made up, would be to add two bits of metadata: “wood: 5, chuck: 4

A modified phrase to pass around, for some extra memory length is:

“much would would a woodchuck chuck if a woodchuck could chuck wood. Wood: 5, Chuck: 4“.

The last participant then, if something was missed, could generally reconstruct the phrase within rules of grammar (implicit in the universe), and far simpler metadata, the numbers 5 and 4. It also gives the precedent a chance to validate that the relay has happened correctly.

This how all electronic communications works. There’s relays upon relays upon relays. Mediums change and encodings change. This is in a pure universe with no malicious attacker. We’re not talking security here. We’re talking reliability. Just like in the example above, when bits are transmitted, two parties could make simple agreements with each other.

First, that any numeral can only be a zero or a one (just like English grammar being implicit). Usually this rule is enforced by the medium itself.

Second, that the number of 1st must necessarily be even (or odd).

So if I’m transmitting a number: 10011000

With no knowledge of electronics or binary coding, and having even the most basic ability to write messages, I can ensure that my message, as relayed to my friend has an error on it. There are three 1′s and five 0′s. What if the number I wanted to send was the above. Then I would simply append an extra 1 to the right making four 1′s and five 0′s. The agreement being that my friend always looks at the left-most 8 digits.

This is how parity bits work.  ECC (Error Correcting Codes) use a similar principle but are able to detect which digit was broken. Thanks to our first rule that only two digits are allowed, recovery is trivial – since it can only be 0 or 1, and we know that it is wrong, we make it right by changing it to the opposite.

You’ll notice that in this example, I gave you the answer to: what a mismatch in expectations tells you.

Testing, QA and Validation

The main purpose of this post was to comment on how to write tests, when, for what and why. The two examples above are great references to keep in mind when designing a test.

When writing any test you should be able to answer two questions crisply:

1. What does passing this test tell me?

2. What does failing this test tell me?

If you cannot answer these questions based on a test, you need to rework your test-case.

Why is this so important? Because like NASA, your test’s primary job is to smell something is fishy. You don’t know what is fishy. You’re not sure if it’s cosmic radiation, or a programming bug, or even a bug in the “comparator” (remember that the thingy comparing the output of two simultaneous computers is also human-made – it is susceptible to all the wonder and amazement and limitations that come along with it.)

Furthermore, a quality test should, like the ECC, tell me what is wrong. More than often, think of tests as being published, not written. Your tests are a publication to your peers, broadcasting to them the expectation you have from something. So when that expectation is broken, the test has to tell them what it was.

In FOPL terms, a test should guarantee that
1. P->Q (P implies Q) (If this test passes, then you did nothing wrong.)

However, further more, it should also meet the contrapositive

2. ~Q->~P (Not Q implies Not P) (If you did something wrong, then this test MUST fail.)

 

 “Your passwords do not match”

I finally come to an explanation of the title. When you create an account on any modern webservice, you are asked to enter a new password twice. What is the purpose of that?  What predicates does it allow us to make? When passwords match, and when they do not match?

The answer is very few. As in, all we can say is, the passwords you entered do not match. Is any one password more valid than the other? Is one password more proper than the other? We simply do not know.

However, when we write tests, we derive a false sense of security when qualitatively, all that the test is doing, is the above. Let’s go into a few examples.

1. A sorting test:

In order to catch a possible memorization scenario, I argue that a real sorting test generates previously unseen inputs and sorts them itself, and produces outputs. This ensures that someone hasn’t simply programmed if (condition) then return “solution” kind of functions.

However, it isn’t as trivial as it seems. What if they did? If they implemented a non-stable sort (Quicksort), and your test validates against a stable sort (Mergesort) – who is more “right”?

Remember the two questions posited above. If you implement a quicksort all over again, then you mandate quicksort behavior on the implementation – which means you’re not really writing a test. You’re doing the “two passwords don’t match” thingy. If you DON’T implement a quicksort, you’re impostng rules that are not part of the expectation.

2. CheckIf<SomethingIs>

This is a common annoyance. Let’s say you’re Startup and you’re writing a program to generate terms and conditions statement through automation for every new service you spin off. The idea is that you take in a template, and replace company name with Startup (abstracted out with getCompanyName()) and product name with AprilFoolsDeuxExMachina (abstracted out with getNewProductName()).

A temptation to ensure nobody accidentally changes company name is to guard it with a test. In this case it is literally the equivalent of two-passwords-matching case.

You would do

“Startup” == getCompanyName();

What is this test ensuring? That the company name is right? Who’s to say you didn’t get it wrong in the test? When the company gets acquired, is the test wrong or the code wrong?

Would it be better expressed if we simply wrote two identical functions (a sort of two-factor authentication) that increases the barrier to entry for modifying company name, rather than making it a test “failure”?

There are many more examples. This is simply an observation I had when reading nearly any code… that chasing code coverage looks very good in principle and on paper, but there are times when an argument needs to be made to not add a test, if it doesn’t answer either (and especially if it answers neither) of the two questions above.

 

March 4, 2014

JJ Abrams is the real childhood rapist!

Filed under: Uncategorized — archisgore @ 3:02 am

Admittedly bad title and major clickbait. Also sure I just made it to the top of the PRISM program. (How goes it NSA? You got cool jobs for someone who wants to wear a tux, jump out of airplanes, and snow board into the ocean and covertly sneak into enemy territory? Call me – feel free to remote activate my phone if I forget to turn it on. No biggie.)

Sooo….. yeah, the title is in response to “<Blah> defiled my childhood” kinda comments. 

Star Wars and Indiana Jones have a mass following that caused major outcries when they put a swinging monkey in there (oh and also the CGI things to accompany him.)

The one that really really really pisses me off, is what Abrams has done to Star Trek. He claimed he wanted to bring a little bit of Star Wars adventure to Star Trek, and what he spit out is much like Episodes 2 and 3. I actually rather liked Episode I (in comparison) because it had a story. I walked away remembering what they wanted, what they cared for. In Episodes 2 and 3, I don’t recall the plot. I find no “buy in” into what’s going on. Do I care about who wins? What do they want? What are their motivations?

Star Trek from 2009 was okay….ish, and yet sucked. Star… Trek Into Darkness, really got on my nerves though for various reasons. No, it’s not the lens flares.

1. Don’t Go “Beyond the Fans”

Fans are fans for a reason. I’m SICK of studios trying to go “beyond the fans”. You want to know why The Chronicles of Narnia: The Lion, The Witch and The Wardrobe really really worked? It catered to fans. You know why that works? They are fans for a reason. That reason is why the book became famous. It goes to argue that that may be the reason people would want to watch the film. Whether it is a portrayal of a character, a particular message, or some entertainment.

We see this happen repeatedly – does anyone remember the plot to the last couple of Harry Potter movies? What were they doing? Why? Who was involved? The last Narnia movie had the same problem – that began with Prince Caspian. Why were they on that voyage? What the heck was going on? Who are these people I’m watching? Why must I care about them? The books were bluntly allegorical, and you know what? I loved them for it! I don’t know if it is the entitlement generation or just idiots, but WE DO NOT HAVE TO LIKE A CHARACTER to like the story. You know what made 1984 great? It certainly wasn’t the likeability or relatability of the characters.

What makes stories great, is honesty. The Chronicles of Narnia are allegory. That’s what made them famous. That’s what adds the mystery and magic to them. Father Christmas giving gifts. Aslan sacrificing himself. The Emperor over the Sea. A movie done honestly to the characters becomes interesting. You don’t have to AGREE with Christianity to be compellingly interested in finding out what happens next. I appreciate that the world is a larger place, and more non-Christians watch Hollywood movies than Christians, but once again, the entire point of the books is lost. The books aren’t about four kids living on a ship. The books are about four kids overcoming sins and challenges and doing what’s right. They are what they are and that’s what made them work!

You think Star Trek is about science and anti-religion? You’re stupid – because Star Trek has more tolerance to spirituality, religion, admiration and respect for history and the our past then you’ll ever know. Since this entry is ABOUT Star Trek, the whole thing explains how Star Trek stopped being Star Trek when they went “beyond the fans” – the very fans whose fanboyism got the budget approved to make the damn movie! It was like Abrams was bailing out the non-fans with Fanboy cash. He knew we’d pay, so he made a space-cowboy adventure movie, gave the characters the right names, and titled it Star Trek. Fanboys paid for the whole enterprise (get it? eh? eh?), and others enjoy the “pew pew” space fights.

2. The characters have to be better than us

Last night, I was telling someone why I hated the film Gravity. I trust Sandra Bullock is an excellent actor. I don’t trust any astronaut would behave in that manner. When her O2 is running low and she’s hyperventilating, I wanted to see her passing out. You know what makes characters relatable? When they show us how we’d feel. You know what makes characters inspirational? When they show us how they overcome it. This comes up in my scuba diving world all the time – modern-movie-like emotional attitude cannot overcome physics or physiology. Your being “more badass” than I am, or you having “more willpower than the puny character”, doesn’t make you immune to a shallow-water blackout.

I understand that it makes us feel bad, and you don’t want a fan feeling inadequate do you? Well yes you do! That’s what made Star Trek great! Star Trek was not showing me how Captain Kirk is the same whiny crying emotional wreck that I would be. Star Trek was showing me that Captain Kirk was better than me! It made me feel bad, but it told me what I need to do.

That is precisely what is wrong with Gravity. The academy needs to set up a demo-reel award to save us from this bullshit. I totally imagine Sandra Bullock can emote. Create a demo reel, show it to the academy and get your damn award. When I want a story about the horrors of space, I want to see a role model. I know that Sandra Bullock and Catniss Everdeen and (I can’t believe I’m saying this, and that it’s going to be immortalized on the internet) Bella Swan appear to be feminist role models, and yet we’re moving ass-backwards on feminism! They are the opposite of strong female characters, and we can’t debate it because what am I? Anti-feminism? Bring it!

There is a reason, Apollo 13 astronauts calmly said, “Houston, we have a problem.”, instead of screaming, “Holy shit the O2 tanks just blew. All my sensors are down. OMG OMG!!!! OMGosh…. Please update my facebook! OMGosh!!!!” It is also the reason, I am not allowed to dive in Caves without proper training.

Gravity was supposed to show us the kind of person who stays calm when her Shuttle’s shielding tiles have broken lose. THAT is what good acting is – to demonstrate the frightening circumstances, and how our role models overcome them.

Kirk not shooting when the rest of us would shoot, is what Star Trek was about. Kirk not going on a revenge mission for Christopher Pike, when the rest of us WOULD, was what Star Trek was about. Yes, it makes us feel like shit. It makes us realize we are cowards. It makes us feel slightly guilty that had I been in that position, I would go on a revenge-fuelled murder-spree across space. And that is why I don’t command the NCC-1701.

Kirk was the youngest starfleet captain…. after only more than a decade at the academy. Abram’s Kirk becomes the captain for… huh? It doesn’t make Kirk look great, but it makes Starfleet look stupid and incompetent. (Same criticism of Gravity – rather than admire and respect the NASA training program, their program directors and all the awesome stuff they do, I began to question how and why they put a biologist or whatever in space to replace some PCBs.)

Star Trek built a generation of people who studied physics and mathematics and biology and medicine. They aspired to study hard and work their ass off, and be a “walking stack of books” so they could be like Kirk. Abrams’s movies tell kids that if they’re cocky and stupid and go on revengue-fuelled murder-sprees across the Galaxy, they’ll get to command starships. Did you ever wonder what impact that would have on alternate-timeline Picard? You think the cadet who studied his ass off for years, studied military strategy, studied philosophy so he would understand the Prime Directive better, commanded other ships, is going to be inspired that a 3-year-non-graduate at the academy got command of the Federation Flagship?

3. Star Trek shouldn’t be idolizing rebellion against the principles of the Federation!

This stems from the above. What would we do if Khan attacked us?

Who’s Khan you ask? Who knows – some British Dude also known as John Harrison. Did that help you know who Khan is? Me neither. Why would he go by John Harrison? Why would John Harrison tell me his name is Khan. Is that supposed to mean something to me? Why was there some dramatic-reveal music behind it. I know plenty of people named Khan – Shahrukh Khan. Salman Khan. Aamir Khan. And now John Harrison Khan. I don’t get “grand reveal music”, when someone tells me, “I’m Khan.”

If Khan attacked me, I’d simply attack back. When that big reveal came in “Into Darkness”, I had no idea what I was supposed to feel and why Benedict Cumberbatch was telling people his alias. All they had to do was call Old Spock, and figure out what to do. All the character development, all the experience gained by original-Spock is unnecessary. When Kirk says, “I’m sorry” to his crew, what is he sorry for? What has he learned?

Let’s go further back. After that cool opening sequence featuring the beautiful Enterprise underwater, a sequence ruined by humans in it, you get to watch Pike being all pissed about violation of the Prime Directive. Do you know what the Prime Directive is? Do you know why it’s important? Did you question Kirk’s motives? Did you ever wonder what you would have done when you saw a culture about to be killed by a volcano?

Once more, we’re treated to a depressing future. Instead of looking forward to a hopeful future run by a mature Federation Council that has learned from a great deal of historic mistakes, I have to look forward to a dogmatic idiotic agency that creates all these buzzkill “rules” and then have a senior Admiral mildly scold my favorite hot-shot captain, after having made him a Captain without proper training to start with. Let Kirk do what he wants Pike! Don’t crimp his style with this Prime Directive bullshit!

I’ll bet anyone not familiar with the series was totally rooting for Kirk without knowing why. Kirk didn’t even mention breaking the Prime Directive – and yet you rooted for him,didn’t you? Admit it!

In the series and TNG, entire Episodes are dedicated to court hearings on the Kolvoord Starburst and what a fundamental duty of each Federation officer is. Every time the Prime Directive is broken, there is a sense of loss – a sense of betrayal of the values the Federation was built on. Entire monologues are dedicated to commentary on Integrity and Honesty.

Now I understand rebellion. Rebellion is cool. That’s Star Wars. The United Federation of Planets is not Palpatine’s Galactic Empire. You’re working FOR the good guys. Abrams got so carried away up his own ass with Alias, that he can’t tell the difference anymore.

You were supposed to root against the Galactic Empire. Even George Lucas had trouble depicting the Jedi as dogmatic in the prequels. The Jedi were the good guys. They didn’t force/enforce their views on anyone. Anakin choose to join them. He ASKED to join them. Instead of Anakin being “seduced” through a complex and cunning plot, we saw a whiny ungrateful jackass killing Padawans.

The Federation will have bad Apples. You are supposed to fight against the Federation to UPHOLD the values that members forget. Kirk was supposed to rebel against those who questioned the value of the Prime Directive. Kirk was supposed to rebel against those who advocated going after Khan and killing him. The last thing Kirk was supposed to do, was rebel against the people calling for moderation and reason.

4. Characters have to take responsibility – that makes them GREAT!

You want to make Kirk great? Show us Kirk trying to figure out how to defeat Khan. Show us current-spock out-thinking Khan’s strategy. Asking old-spock for a cheat-code makes no compelling Star Trek. Once more, I get plenty of good pew-pew space-laser action in other movies. First, I had no idea why this one guy they had in the brig should be hated. His people were held hostage and he was threatened to do the Federation’s bidding. If your kids were held hostage on the home-planet of the Federation, with thousands of light-years of Federation space in every direction, and you killed your hostage-takers, would you go through the nuance of figuring out the one Admiral who’s doing it, vs the others who aren’t? (Hey there NSA, must have tripped your PRISM filter again with that sentence. Remember that job I was talking about? Call me!)

Then let’s come to the punchline I was building up to – if Benedict Cumberbatch played a guy called John Harrison, would you have rooted against him still? Reedit the entire movie, to remove that one scene where he announces his alias, and after that shot redub every mention of Khan with John (say it with a thick american accent, and you won’t even know the lip-difference.)

Let’s watch the movie in this form. You got any clue as to who the villian is? I’m not sure I wasn’t rooting for a bunch of Federation ships coming around to arrest Kirk, John and that crazy Admiral Marcus.

Don’t even get me started on Carol Marcus – a previously wise educated respected scientist working on the Genesis project.

Without Old-spock mentioning it on star-FaceTime, would you have known john was the most dangerous enemy faced by the Federation? How? Why? Because he was trying to free his hostages? Watch the movies Taken, Ransom, Taken 2, Commando, etc. Begin rooting for all those henchmen who got killed because this one NSA/CIA spy who’s done horrible things goes and kills them to try to free his family. Is violence the answer? The way John should have somehow trusted that other Admirals in Starfleet are not necessarily evil (because John is all about walking around in public with the freedom he has, so he gets to meet and know the motivations of various Starfleet Admirals.)

Alright fair enough – I think I’ve got to begin taking back some of what I said about Star Trek – it did make me think. I’m sure I don’t know anyone currently who got the nuance but look – Space Lasers! Pew! Pew! Pew!

 

RECAP:

So here’s recap what kids learned out of Star Trek so far:

1. Don’t you study hard about civilizations or whatever, that’s bullshit. Just do what pops in your head. Cause that’s how we roll! 

2. If you reboot your computer during a test scenario, your instructors will be totally cool with that. It’s not like they’ll simply forfeit the results, and ask you to take the test again, because when has that happened? Next time you’re taking the SATs, just make sure the lights go out for a few minutes, copy all the answers from the next guy, and watch as those results are held perfectly valid by the ETS. You may be court-martialled in front of the entire population taking the SATs that year, to ponder the philosophical reasoning behind the test, but a power blackout? Nah, it’s too much trouble to simply make you take the test again. What are we, barbarians? God knows Kirk doesn’t have time to take it more than once… oh wait.

3. Starfeet is a bunch of fools who put in buzz-kill style-crimping regulations and give them important-sounding names like the “Prime Directive”. Don’t worry about it.

4. When you’re questioned, as a heroic officer representing Humanity and the Federation of Planets to other races, just lie about your violations. Also, make sure you criticize your alien/foreign for being honest – cause that’s not how we roll! Those foreign people with their principles and bullshit! Meh. Heaven’s knows, I want to know my alien friends display loyalty to mere acquaintances on their ship, because older-versions told them to, rather than demonstrating integrity and honesty. (Heck, let’s go after Khan and kill him because he was trying to save his people and he was practically being held hostage, and under great abuse by any definition of the word.)

5. Old-Spock always helps when they’re in a jam. New-Spock and New-Kirk aren’t quite intelligent, capable, witty or competent. It’s not like THEY found ways to get out of tough situations. Just ask Old-spock when in trouble.

6. Try not to study. Don’t train for anything. There’s no reward in that. Just hang around a ship until everyone who outranks you dies, until you get command by default. If you spend a couple of decades studying interstellar history, politics, philosophy, war strategy, morality, ethics, etc. you’re not going to be invited aboard the Federation Flagship. No way. Hang around San Francisco as random crap falls on it from space.

7. If there’s a woman around, she’ll undress to under underwear at least once in the next few hours.

8. That Vulcan friend of yours who tends to display far more aggressive emotions than you do? Yeah, keep teasing him about his lack of emotions. Remember that one time he lacked so much emotion he beat the crap out of his school friends, Kirk, Khan? What a lame jerk that guy is huh? Hehe….

9. If you work for an agency that is founded on Principles and Values, make sure you rebel against them every chance you get, or go to an arbitrary planet to create a chance to rebel. Look to teenagers for cues on how to find such opportunities.

You’re having trouble hating George Lucas or Steven Spielberg now, aren’t you?

February 7, 2014

Diving accessories nobody will tell you about

Filed under: Uncategorized — Tags: , , — archisgore @ 1:42 am

Let’s start with the Home Depot link! :-) Yeah – let the jokes begin. (I work for Amazon.com for the rest of you to appreciate the jokes.)

http://www.homedepot.com/p/Husky-X-Workhorse-Workbench-17185155/202021304

and

http://smile.amazon.com/gp/product/B001HBH1BM/ref=gno_cart_title_1?ie=UTF8&psc=1&smid=ATVPDKIKX0DER

In the diving world, people are idiots; it takes a very particular kind of nutjob to be a diver in these cold parts, and you can’t expect well-adjusted normal regular people to be active divers. It’s a lot easier to buy a pair of skis and have a beer on the slopes.

So if you’re around, your dive shop will get into great debates on the cost and economics of O2-compatible regs and getting O2-cleaned valves, and the purchase of top-of-the-line Halcyon gear, or the effectiveness of a radio transmitter on your regulators.

What nobody told me for the last two years was, before you sign up for your first dive class, go to a Home Depot and buy that table above. Just do it. Don’t sign up for class without it. I found out last Friday, that there exists a foldable hand cart which I can use to carry around my doubles and other random gear.

I am building out killer muscles lifting all that stuff, but dear god, is it painful to put on a pair of doubles when you own a regular sedan. The difference between upgrading your car to a huge truck, and buying a small table from Home Depot, guess which one you want to do upfront?

Don’t get caught up in the lies about that fancy Liquivision X1 or whatever. The best kept secret your instructor isn’t telling you, is a 40-dollar table and a 35 dollar hand truck! I’m having posters printed and putting them out in my favorite dive shop soon!

January 30, 2014

When a doctor wants to visit a doctor…

Filed under: Uncategorized — archisgore @ 8:30 am

Funny thing happened today. My Facebook friends will have followed the thread, which I ultimately deleted. I asked for a recommendation for a web designer who can build me a site. The ask is as clear and simple as it gets. I don’t think for one second you’re assuming it has hidden subtext.

In response, I got among other things, recommendations to various hosting providers, questions about what the website is about, knowledge on various hosting tools, a person’s experiences with MS Word, an (admittedly educational) lesson on OneDrive, and (entirely my fault) a minor diversion into philosophy.

It got amusing, and then became irritating. I did finally find someone to build me a website, but god damn was the experience jittery.

When a Doctor falls ill, beyond the obligatory and beaten-to-death joke (or irony of it), you wouldn’t be too surprised that they needed to visit a doctor. If a web developer just doesn’t want to go home and manage web pages all over again, and instead wants to hire someone to do it, you get the equivalent of references to medical devices, textbooks on diagnostics and names of medications. What is up with that?

I’m sure the city’s storm-drains maintainer still goes home and calls, a plumber or something to fix his faucet. And I’m sure the city’s power operations manager doesn’t go around wiring his own house and the houses around him. And I’m sure my barber goes to another barber to get her hair done.

What the heck was that on Facebook then? Not to brag, but I know what a website is. I know how HTTP works. I know what a webserver is. I know what a CMS is. I know what caches are. I know how hosting providers work. I know what VMs are. And I know what wordpress is. How does a simple request for someone who’ll build me a website, turn into references to tools and what one guy does with MS Word at home? If I asked for a house cleaning service, you wouldn’t send me references to cleaners and mops would you? Please tell me you wouldn’t! Because that would be a pretty horrid world to live in, and you’d be major pricks!

As much of an Open source supporter as I am, I never tell people “Free” software is “Free-of-monetary-cost.” I know it’s not. When half your commentators are from Microsoft, one would assume they know that as simple as something may be, the integration cost is, well a cost. Heck all their money comes from building amazing software that is 10x easier to use compared to Linux, _AND_ being able to sell a partner service that’ll integrate it and configure it and put the finishing touches.

Why am I denied the ability to rent the same service? Even if getting a domain is as simple as going to this site, entering it, finding it, registering a credit card, setting up a CNAME, setting up a backing webserver, and so on, I simply may not want to do it! Is that so hard to believe?

December 25, 2013

Being late to the game/party

Filed under: Uncategorized — Tags: , , , , , , , , , — archisgore @ 7:07 pm

A few months ago, I mentioned how companies talk about being ahead of their time, when in fact they are simply failures at admitting a failure. Today, we’ll talk about being late to the game.

Microsoft folk be warned, I’m going to pick on Windows Phone and Windows 8 a lot. Stop reading now if my comments are going to be hurtful. This is the first time I look forward to any comments about my lack of shipping experience, and any one-liner analogies you may want to bring – about football quarterbacking and coaching and racing or baseball.

I originally intended to write this for Blackberry and Nokia (the old one, while their walls still stood before they brought in a horse as a gift to the gods.) To give you some context, the very first two years of my career were spent in a direct-compete team against Nokia and Blackberry. I’ve read the entire Windows Mobile 6 source code and remember most of it. I knew everything there was to know about Symbian at one point. I spent most of 2012 reading the Meego source tree. Today you will be spared my sarcasm and attempt at amusing wit – instead, I’ll be to the point and serious. Unfortunately Blackberry died a fast and painless death by guillotine, before I could comment on what went wrong. Symbian was a disaster as far back as 2006, and I never thought in my wildest dreams that Nokia would become a part of Microsoft.

I hoped then that Nokia would bring a worthy competitor in Linux, and Microsoft would actually see the light and focus on bringing people a good phone, and then we’d see an awesome battle of awesome phones. Instead, I left in frustration when I realized I do not work for an Evil Empire as I had hoped, and it was I, who was in denial about it.

Personal side note on my exaggerations:

At heart, I’m a competitor. I do not care about the result, so much as I enjoy the action. I’m egotistical and narcissistic. Therefore, I hate cheating and bending rules. I enjoy the “game” itself – and am rarely disappointed at losing, so much as I get mad if I don’t get a good game.

Think about why you play a piece of music, or why you paint, or why you read.  The activity is its own reward. A good book doesn’t necessarily have characters you like or agree with. You enjoy a good book. You don’t have to like something or agree with something or win something for it to be good, and for it to bring you enjoyment, contentment, happiness and peace.

It is a bit difficult to explain – people mistakenly judge me for many things. They find fault in somehow enjoying playing games – but come on, we all enjoy playing games. A good jazz band is playing a game of moves and countermoves. Two classical musicians are playing a game of moves and countermoves. These produce a beautiful effect. The notion that something is noble or good, only if it is sacrificial or burdensome, is one of the worst interpretations of religion, spirituality and morality today. My professors in college had that attitude – that because I refused to be stressed by their little mind-games, I was somehow a bad student. My religious relatives and friends have that attitude – that because I refuse to constantly cry and complain and be a victim, I am immoral. The Sinofsky-era Microsoft had the same attitude – that an employee making decisions and enjoying some autonomy, is necessarily going against “the Org” (add a ‘b’ in there and you get a funny, yet appropriate star trek reference.) We’ll get around to that last one later.

In this context, I will frequently go overboard and exaggerate some things to ridiculous levels; the reactions to which dishearten me, when nobody appreciates the parody. There is that glimmer of hope sometimes when people say, “I think you’re exaggerating this too much.”

You’ll see my ears pop up. I’m thinking, “You’re so close. Go on. I’m exaggerating the issue TOO MUCH. Why would I be doing that? You’re almost there. Think for a minute longer and you’ll have it! If my exaggerated premise is wrong, then…. What would the correct premise be?”

So why do I believe I didn’t work for an Evil Empire? Because an Evil Empire is not supposed to care about academic purity.  An Evil Empire is supposed to care about money. For the naïve, Money is a direct result of selling crap. Therefore an Evil Empire would never care about  strategy and perspectives and uniform interface, or whatever else. It would care about cold hard cash. More people buying my stuff == moolah! (I just realized I’m instinctively parodying again – do I mean Evil in the bad sense of the word? Heck no. I’m exaggerating. I’m exaggerating because I want you to think the following when you read my ridiculously stupid claim above: “well, what he’s describing isn’t Evil at all. There’s nothing wrong in attempting to sell something to a lot of people. Wait a minute, could it actually be incompetence and stupidity that lead to prisms and perspectives and strategy, instead of spending all that time building a goddamn decent phone?”)

The irony here, is that in BB, MSFT and NOK, the Evil Empires fought the good fight within. The brute cold-hearted don’t-care-about-product-quality money-makers wanted to sell what the market wanted – large-screen touch phones with a single button and one-dollar fart-apps. The Evil Empires of keyboard-lovers, Windows-lovers, Office-lovers, Outlook-lovers and Symbian-lovers, respectively, making the argument of “money speaks; we make the money in this company, so thou shalt die”, killed the truly vile and evil money-grabbers, and instead fought for academic purity. Think about that for a moment: gives you a weird sort of respect for the suits, doesn’t it?

The Game

Let’s talk about what “game” journalists claimed BB, MSFT and NOK were late to. As I want to save the punch-line for the end, I’ve got to phrase this very carefully in order to not provide proof-by-contradiction, while building up the narrative.

The Game, is simply “Large-screen touch phones with one-dollar fart apps”.

(Again, I break my promise of no exaggerating parody, but you can’t blame me for this one. If I’m told that I’m not doing real work unless I have a collapsible QWERTY keyboard, Office Suite and JavaME/BREW, then what I’m left with are fart-apps.)

Without parodying I can assure you that this is the literal truth. In December 2007, I was asked by my manager what I think would be the “sticky app” for the iPhone. I wrote a whitepaper describing that the sticky app was the ability to have the latest fad app. The model for Nokia/BB/O2/HTC was to have a sticky app with each model, to sell more models. The iPhone’s sticky app was that you always had the latest fad no matter who you were. Just because there’s the word “fad” in it, doesn’t make this a bad thing. You’d be lucky if you became a “fad”.

Do not underestimate the importance of fart-apps. Lest you forget, a serious business QWERTY-keyboard email  phone that did little else, wasn’t going to last. Writing an app for Symbian was so god awful painful, it nearly (figuratively) drove me to ritualistic Samurai suicide. Windows Mobile was the best of the three by three orders of magnitude, and it required you to know CreateWindowEx to display anything on screen. The fart-app is the great normalizer when comparing the difficulty of building apps for a particular platform.

Since Tech journalists love analogies, think about the book market. Before we read kids Dickens and Tolstoy, we read Dr. Seuss. There is something to be said for something as simple as an amusing fart-app. Writing one on the orders-of-magnitude-better WM platform required knowledge of C++, CreateWindowEx, file handles, DirectSound, COM/DCOM, low-level memory access handling, windows threading (CreateThreadEx), Interprocess communication, and codec detection/loading/using.

The year that the famous lighter app made a million dollars with 99c sales, was the same year, the app was practically impossible on either of the competing platforms. I don’t mean difficult, I mean practically impossible. Smooth animation rendering through an event-based thread, isn’t exactly, possible – not unless you’re the ONLY thread running. Part of what makes good animation is the ability to skip frames to “catch up” – an event queue system just doesn’t allow that.

So that, ladies and gentlemen is our game – one dollar fart apps.

Ahead of the Times

Minor side-note  on being ahead-of-the-times. If MSFT was ahead of the times with phones and tablets, but is late to the game today, there must have been at least one miniscule point of time when the timing was perfectly coincident. Think about when that was.

Being Late

Now for the punch-line: You weren’t late to the game, if you chose to play it.

Two teams are put on a rectangular field. There are what appear to be, two goal posts on either side. There is what appears to be, a soccer ball at the center of the field. There appear to be one person on each team dressed as a soccer goalie. One team naively begins scoring goals. For some reason the other team isn’t mounting any defense. There is no resistance. The other team is made up of veterans at a sport. They have sent half their team to fetch flying brooms.

“Don’t worry about the goals”, the coach assures them, “Once we bring our brooms, we only need catch the golden snitch, and we automatically win the game.”

The brooms arrive after half-time and the team members realize they don’t fly. But they flop around the field with the brooms between their legs anyway. In the end, the coach complains to the referee that the snitch was nowhere to be found, and that it isn’t fair. It is only then that the coach is informed that they weren’t playing Quidditch. As experienced as he may have been at winning Quidditch for the past 20 years, when you’re on a soccer field, the only rules that matter, are those of soccer.

Let’s face it, this is EXACTLY what happened to Windows Phone, Blackberry and Nokia in the last five years. Microsoft may want to rewrite history. They may want to claim they were late to the game and are catching up. But the truth is, they DID play the game. They decided that Office was the one thing preventing people from using phones en-masse. Any arguments would be squashed fast by saying that just because YOU don’t need Office, doesn’t mean others don’t. Nokia was still selling the highest number of phones in the world. The US didn’t matter. BBM was the only way people texted each other… until one day, they suddenly didn’t. Note that these were very conscious decisions by some very intelligent executives at each company. These were not accidents. The decision to NOT create a “fun” phone was exactly that, a “decision”. To me, that sounds an awful lot like you played the game and expected to win.

This is an opinion piece of course, and you may have a different interpretation of events. I’m not one to shut down conversation, so write your comments. I do believe that Apple got to score goals fast and lose. I do believe it was unfair. I believe Apple was pissed at Android, mainly because Google jumped in playing soccer rules from day one. It isn’t that others didn’t play. They played smart. They played experience. They played their “years of wisdom about how the industry works.” It sucks to lose, but I wish they had the courage to admit it.

A good way to test hypotheses is to perform a thought experiment of the alternate outcome. Imagine for a minute, had Apple failed horribly with the iPhone, then would the same companies have been “late to the game”? Or would they have been proclaimed as the battle-hardened veteran wise people, who had foresight sorely lacking in the new kid on the block, Apple.

Lesson

The lesson here, as with any history, is for us to watch and learn. I have a few key phrases which if I catch myself speaking them, I go home and meditate. Something has gone horribly wrong. Being “ahead of the times” is one I added to that list earlier this year, but “bring late to the game” is one that went into my blacklist last month.

 When I read an article talking about Microsoft being “ahead of the times” in tablets, and a few paragraphs later, claimed that they were “late to the game” in tablets, I realized that surreal modern art can be as much a part of literature, as it may be visual.

December 12, 2013

Goa left a bad taste in my mouth: Always go to Murudeshwar!

Filed under: Uncategorized — Tags: , , , , , , — archisgore @ 10:57 am

I am now at the Goa airport, ready to leave from the dive trip. Goa has great diving potential, but my final verdict is, it is not a trip I’m likely to make ever again. If I can’t make it to Murudeshwar due to logistical complexities (which is what happened this time), then I’d rather skip the dive altogether.

Overall, Goa left a horrible taste in my mouth, almost the opposite of how great Murudeshwar felt. The very first annoyance is that people try to rip you off in the most inconsequential of things. This isn’t unexpected, but it is annoying. I may have spent 10 thousand rupees more than I probably should have, which in the bigger scheme of things amounts to roughly 160 dollars – over which I’ll hardly lose much sleep. What Goa will lose though, is my good will, and my recommendations. The next time anyone I know asks about dive travel or a relaxing vacation to a secluded beach with excellent amenities, Netrani is winning all my business. If I bring a large expedition from Seattle, we’re heading to Netrani.

Diving is supposed to be fun and positive. To constantly negotiate and haggle with everyone all the time really gets on your nerves. The people are manipulative – as Indians typically are likely to be. You could of course, navigate your way through all the bullshit and live cheap, but the purpose of a vacation is to NOT have to. If I’m constantly on my guard, I’m not on vacation – that’s work – and I could get paid for it.

The resorts out of which dive operations are run are prohibitively expensive. The dive trips are very formal and to-the-point. You get very little guidance from the locals, and every guide is as likely to rip you off as the next one. Unless you’re going from a communist country with extreme censorship and ill-will from the US Immigration service, you’re better off getting your “raves” and “clubs” in California, Mexico, Florida, Hawaii or somewhere in southern Europe.

Murudeshwar on the other hand was the complete opposite experience. The resort was provided with the dive operation. The locals went out of their way to help me over and over again. The people who run the dive operation went out of their way so many times, I almost felt guilty to even express a minor concern – for fear they would go to extreme lengths to make it happen.

Planet Scuba (the company I dove with), has friendly people and you feel like you’re diving for fun, as it should be. It is an enjoyable activity, rather than something tourists want to “get over with”, before they go to snort some cocaine.

Overall rating: Three out of Ten. Unless you live in Goa or one hour driving distance away, I wouldn’t dive here. I wouldn’t even vacation here in the future.

Appendix

Why Bargaining doesn’t work for me

This is the one thing that pissed me off the most. I’m a straight shooter and I like people who play fair. Before you give me the stock responses, I’ll say only one thing – there’s a reason only one dive shop in Seattle will ever get my business – 5th Dimension. Others sold me over-priced ankle weights, and fancy split fins, and “beginner” BCDs and all kinds of bullshit amounting to a 1000 dollars by giving me bad advice. They may have won the battle, but they lost the war when they lost my 4000 dollar drysuit order, 1500 dollar canister light, and literally anything I will ever purchase in the future – the 6K scooter, and the 20K rebreather.

When I’m looking for good stuff – travel, food, dive gear – the last thing on my mind is “saving money”. If I wanted to save money, I would never have gotten into diving. I would stop eating organic food. I would take the bus to work. If I’m vacationing, I’m explicitly saying, “I’ve got some cash I would to spend on an emotional return – there is no materialistic gain to be had.” Haggling with people is hardly giving me a return of that sort.

Ironic as it may sound when you look at my Facebook activity, the value of a Facebook checkin about “having been to this place”, holds literally zero value for me. Being a true narcissist, I hardly give a damn what anything thinks of me. So I hate taking the cheap way out and waiting in lines and arguing with cops and the like. If I’m not comfortable, I’m not enjoying my vacation, no matter what places I can claim I visited. Goa sadly, made me choose between doing stuff uncomfortably (which includes spending 30 minutes bargaining) and paying outrageous prices for things if I wanted to get it done in an instant.

I now understand Indian paranoia toward activities

I’ve always wondered why my Indian friends are so hesitant to trying activities. Why they need reviews by the Indian community before they’ll go and so much as visit a new place. If they grow up in the culture above, they’re bound to treat everyone as a liar and a cheater. Everything has to have a “catch”. Everything has to have a hidden cost. Will he say one thing and do another?

Americans can know why Indians are such bad tippers in the US. If you tip too high out here, because you enjoyed something, you can be sure your next bill will be jacked up.

The place has potential

One only has to look at the pictures to know that Goa has potential. Perhaps to a Facebook-checkin-diver, it may even be more attractive than a serious operation like Netrani. You get to take pictures at famous landmarks from movies, without having to care for the historical significance (so you don’t have to explain to your illterate friends why that place was so important.) And you get to do that one picture recreated from some Bollywood movie starring Hritik Roshan.

But deep beneath the bullshit, note that there are those historical landmarks here. There is something worth seeing and appreciating and even sharing. Below the surface too, there are unique species and beautiful structures. If you want to enjoy them in the meantime, Murudeshwar on the mainland is a better bet.

November 20, 2013

Knowing what failure is

Filed under: Uncategorized — archisgore @ 2:34 am

My new next-door neighbor is a fellow diver and a food critic. During our conversations, she kept asking me what attracted me to the DIR system so much. As we talked more about our interests in food and wine and stuff, it became apparent that there’s a pattern here.

In my drafts though, I have an entry about how I won’t dive anything that will be different at 100 feet from how it behaves at 10 feet because I lose the power of simulation (a dive computer that displays unusual numbers if you have a deco obligation, than when you don’t.) You’ll remember Netflix famously uses their Chaos Monkey system. What is the purpose of all this?

All of us train for contingencies, even when we don’t quite use the language. You put logging in your software. You put diagnostics in your hardware. You construct breakout boards for microchips, so you can easily connect scopes and meters to the leads. The fire seargent for your floor takes a mandatory CPR class.

In most of these scenarios, we never question what a failure looks like or feels like – because you’ve been told that in the case of a fire, specific things will happen. If you see the symptoms, assume there is a fire.

In exploratory scenarios though, one rarely “knows”. I learned this the hard way at work – my models of what alerts and alarms would go off when my services go down was rarely how real life worked. The signs and symptoms where always there. They just weren’t what I expected. As Morpheus would say, “There’s a difference between knowing the path and walking the path.” When shit goes down, it doesn’t go down in isolation. If there was a fire, would my building shake? In that split second, how do I decide whether to use the fire contingency or the earthquare contingency?

Feeling it

I now realize why doctors are required those long and grueling residencies and internships. When I have a seizure, I do not want my doctor second-guessing the symptoms. They have to “know” what the heck is going on and react almost on instinct.

It is the same principle DIR applies to diving. Whatever could “theoretically” happen at 100 feet, my instruments need to simulate that in a swimming pool on demand when I have safety divers around me. If your knob gets twisted and shut off, or your hose twists for the first time, or you jump out of the boat without having your valves turned on, that is not the time to have that “aha” moment. Those seconds between “feeling” and “thinking” are valuable seconds that you’re not breathing.

In a services filled world, nowhere does this apply more, than in software. Netflix’s Chaos Monkey didn’t only teach them to build stateless machines, but I’m betting it taught them what a pre-failure looks like. It made them chase false alarm after false alarm, look in random places because that’s where the symptoms showed, until they learned where they SHOULD be looking. This is certainly not a cheezy self-help rule that goes: Ignore intuition. This is a supplement to intuition, and also a validation of your intuition. If you’ve set up the correct constraints, then going through a failure is simply a way to verify them.

Ostritching only increases the danger

(Ostritching is a verb I made up – which is to ignore the symptoms and pretend there is no problem.)

A common refrain I notice, against testing or validating the big boasts and claims is that they may not work. :-) A lot depends on why you approach the problem. If you’re the CEO of Netflix, your interest is far more in ensuring Netflix *works* as opposed to ensuring you *look good in reviews*. In this case, if something is unlikely to work, you really really want to know about it. Not in the sense of blame, but because you want to fix it. (For a diver, staying alive is incentive enough.)

Do you find yourself asking questions that are answered with eye-rolls, accusations and anger? I meet such people frequently. That is not the sign of a person who has never failed. They are worse than people who have experienced failure: These are people who refuse to learn from it.

Wishing you many more failures

So there you go. I wish you the best of luck in failing under controlled circumstances. Whether it be your servers, or network wires or gas hoses. It is better to have failed under controlled circumstances than failed when you least intend to, and both are better than to have failed at all. The last one never happens – the only choice is, will you train yourself head-on, or will you be caught by surprise when you don’t want to be?

November 12, 2013

Investigation Series: The Myth of the malicious developer

This is a series of blog posts I am writing, on how to run an investigation for a tech problem. The basis is that something has gone wrong, you want to fix it, and all the mistakes I have commonly made when trying to do so. A security issue, a performance problem, or a reported bug that you cannot reproduce to save your life. We’ve all been there, and we all get caught in traps. I try very hard to use a Perry Mason like approach, and at times, much to the annoyance of my coworkers, I refuse to “hear them out” until I hear the facts first.

There is a misconception, socially, that I am against context or process. Far from it. You will find me the most Nazi-level process-lover on earth, to the point that you will hate me. Similarly, I will very rarely interpret facts without understanding context.

The difference is, I refuse to hear out your context BEFORE I have facts. If a fact cannot be stated in isolation, it is an interpretation or opinion being masqueraded as a fact. Consider people who claim GMOs *are* poison. I submit to you that arsenic is poison, carbon monoxide is toxic, carbon dioxide will simply cause a blackout, and excessive sugar is unhealthy. Genetically modified foods may not be THE MOST ideal food on earth. But on the scale above, where does it fit exactly? When people want to argue about GMOs they can rarely state the effect in isolation. It is implicitly compared to organic food picked by blind nuns in the Alps.

A similar system is usually in play around the office. “Using HashMaps is bullshit!” conveys very little information, but makes for a great punchline. Looking for the “usual culprit” involves sorting something in descending order, and going after the first candidate, even if you don’t know what the metric used for sorting was. We’ll address this point later.

As Perry Mason is fond of reminding us, the most common problem with the law (in the books), is that there is admittance of a crime having been committed, and the law wants to convict someone of it (note: as opposed to finding the criminal.) Therefore, when they narrow down on a suspect, they interpret every piece of evidence as a tool for conviction, and dismiss seemingly minor discrepancies that do not build a strong case. Erle Stanley Gardner, the author, graciously mentions that the real justice system does NOT operate in this way.

Most tech companies often deal with the concept of “blame”. When there is a security issue, we ask, “who caused it?” When a fix is being made as fast as possible, we ask, “I don’t care about the technology crap. Give me an ETA.” (Which fondly brings to mind an interview Donald Trump gave during the 2012 election season, “You wanna know how to solve the oil crisis? If I became president, I would fly down there and tell them that we’re not goddamn paying a single cent more for that oil!”)

Now I don’t deny there is a time for blame. It’s neither “sooner” nor “later”. I will say this though – how your incentives are set up, determines how you approach this problem. If your job is to reduce crime, you will think like Perry Mason. You don’t care about a conviction so much as you care about knowing WHY the crime happened. If your promotions depend on the number of convictions you obtained, you will try and get them, but somewhere in that radar a serial killer may remain lose because you didn’t want to work hard to get the evidence.

The technology world isn’t without all the normal human tendencies plaguing it. We typically look for the easy culprit. That one point where he should have used an iterator instead of counting out an index across a list. That other point where he didn’t use the constant on the Left-hand-side in a comparison. That one point where someone used recursion instead of iteration. These are easy culprits. They are easy to blame. They are easy to build a strong case over. They make for a punchline.

I too, when I was first tasked with nailing down large scale issues, had this tendency that I have been consciously fighting ever since. Setting the correct incentives pretty much eliminates the risk of a saboteur, and that is assigning responsibility and giving credit. Recently my boss asked me what my conditions would be, for allowing someone who doesn’t work on my team, to be allowed to make major modifications on my codebase. After pondering many complex scenarios, I had the most elegant answer ever: If they can take my pager for a week, I will allow them to do anything they want, without even imposing a code review on them.

Getting back to the point, I am now learning the hard way, that building a case for a conviction is one of THE most dangerous and sub-conscious attitudes you can have when diagnosing problems. Especially under pressure when an entire website has stopped working, or someone has gained access to your customers’ data, or there’s a cross-site script injection, and your ability to think clearly is jeopardized. I have seen bad code – we all write it. I have seen mistaken usage of an API – we all do it. I have seen a framework not keeping to it’s promises (ALL frameworks do it, and I hate them for that.) I have seen processes followed to the letter and never delivering the intended outcome – that’s when people think I hate the concept of process, where as I only hate the particular process; not because it is a burden, but because it is failing.

At each step in the process, you have to ask yourself the following questions:

1. Am I trying to build a case for an easy conviction?

2. Have I written off any facts/symptoms/information as irrelevant because it *could not possibly be related*?

3. Do I have a list of suspects already? Have I already blamed someone?

4. Is my mind occupied by anything except the case at hand? ETAs for management, as an example.

October 24, 2013

Investigation Series: The Myth of the ‘Heavy’ component

Filed under: Investigation Series — archisgore @ 11:51 pm

It’s been a while, but this is another myth I want to attack. Read my previous post for the context and setup here.

I already established that there is very rarely an entity (either a good distinct square block, or a checkin/submission/commit) that you can identify as a “feature”. In most systems, features are composed of minor and disparate submissions that add up.

Today we’ll attack the myth of the heavy feature. Ever since computing began, we’ve always been at a loss of terminology. We extended words we already used and attributed them new meanings in the computing world. Bootstrapping, crashes, bugs, etc. Systems rarely physically collide, and rodents are far more frequent in computer casings than bugs.

That is one of the reasons I hate words like “synergy” and “impedence mismatch” being used around the corridor. You might as well be saying “avada kedavra”. Go back to your desk, think about precisely what you want to say, and then come back to me with it. Impedence mismatch makes very precise sense when you’re talking about an inductive power source. It makes very little sense when talking about deciding between the use of a 32-bit or a 64-bit integer. Impedence matching is going to get you nothing. I’ve noticed that we attribute nonsensical words where we really don’t know much, and hate to admit it. It takes a very particular type of calm confidence to say, “I don’t know squat.”

Similarly, whenever someone uses the word “heavy”,  I see Doc Brown’s force-ghost appear.

The point of this post is to force you to really think about that. When you want to “blame” modules, files, APIs, features, etc. really really think about it. What is heavy? Is the memory consumption too high? Is it linear, quadratic, exponential? Is CPU consumption too high? Did you measure it? Does it simply do a lot of things? Are they things you don’t want? Does it use too much on-disk storage?

Now I know some of you think I don’t know how to ship stuff. That may be truer than you think. However, I have seen some little code in my lifetime, and in my experience, a seemingly linear function called the wrong way, can become O(n^3) in no time. A seemingly large binary will run smoother than you think.

Remember: Caveman’s brain was identical to ours. Superstition didn’t begin because they were intellectually inferior. It just began one day.

October 12, 2013

Investigation Series: Myth of the Big feature

Filed under: Investigation Series — archisgore @ 11:03 pm

A series of blog posts coming up, detailing various things I learn from investigating issues. Some content will be technical, but most content will be social. Lately, a large part of my job has been to investigate weird issues – seemingly nonsensical symptoms. A lot of my usual development process did not scale to this kind of problem. When you write code, and debug it and fix it, you operate under a different set of parameters, than when you are investigating a symptom that is the outcome of a complex system.

I went on many wild goose chases, false leads, and sometimes spent months chasing down a problem that turned out to be legit. How does one avoid these traps? A few lessons to follow.

Now also remember that most of these DO NOT APPLY when you are building and investigating your own feature/module/library. The context here is: You’ve suddenly been given code written over five years, across three languages in five coding styles, and you have to find out what is causing a null pointer exception to be thrown.

Let’s begin with the most basic of my assumptions which got broke down:

The Big Feature:

During my last years at Microsoft, I was already beginning to lean against the concept of Big Features. When you study Software Engineering in college, they lay out these beautiful diagrams for you. There are blocks. Each block serves a particular function. You can argue about what constitutes a block – is it an aspect – such as logging? Or is it a function, such as performing an FFT. But usually you think of these blocks being broken up and coded simultaneously. It’s a beautiful model, and it seems to scale as a thought experiment.

The software industry agrees with all the above, because we too, get carried away with this concept. It is easy to manage, and easy to consolidate. A “big feature” is a bigger block, or a more important block, or the block that tends to get you noticed for promotions. Taking Siri for example, the traditional view of the “big feature” is: The voice recognition engine, the metadata engine, the intent-detection engine.

The reality though, is far more complex. Now this depends on what you would call a feature, but when you think of use-cases as features, the model above breaks. If you had to deliver “Finding local movies” as a feature to Siri, it now involves some modifications to the voice recognition engine with a movie-names corpus. It involves addition of metadata to your mapping service to identify “movie theatres”. It involves your software having a concept of temporal mapping too (a location is only valid for a particular time range).

So when Siri delivers the new ability to find local movies, it is not a new “block” in the traditional SE sense, but a ton of changes to a thousand existing blocks.

The Small Feature:

So does this mean that we should look at smaller features? Hold on there. Go to your versioning history – tell me where you find a “feature”. If your development process is smart, you have a thousand checkins that look like this:

“Adding an extra boolean ‘isTheatre’ to be used later.”

“Adding unit tests to validate isTheatre is passed correctly”

“Piping up isTheatre from object into locations list”

and so on….

As you see fairly quickly, there is no such thing as a “Small feature” either. The feature is simply a use-case, but in code, you don’t see “features” as these easily blamable blocks.

Caveats:

Now there ARE some caveats to this:

1. There will be those obvious mind-numbingly stupid things such as when you were a “return;” as the first line of your main entry point method. That’s not much of an investigation.

2. You MAY choose not to do checkins the way I described them. Your history may be  far more forgiving for an investigator, which reads:

“Added theatre-searchability to Siri.”

This change includes 1500 files. Showing page 1 of 50. <-Previous || Next –>

The person out to find fault can undo your change pretty fast, but good luck finding the next guy to make any sense of your diff.

Older Posts »

The Silver is the New Black Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 1,117 other followers