Archis's Blog

September 14, 2011

Windows 8

Filed under: Technology — archisgore @ 12:27 am

Disclaimer: I work for Microsoft. Full-time. Apply all your bias/prejudice/loyalty/whatever filters while you read this. Also, I have intentionally avoided getting “insider info” on Win8, which means what I know at this point, is much less than what the general public knows.

Most of you are aware of my lack of love for many things Microsoft. I am happy with what I saw today, and I am in love with what the Windows team has built, and how fast they built it (24 months ago, we had Win7.) Any programmer worth his 2 cents has to know how tough that is.

When I first joined in 2006, I joined in the middle of what was heating up to be Vista. It was a different time and age in the company but there was certainly energy. I wouldn’t say much else, but there certainly was energy. A Windows that was 6 years in the making. A lot of new stuff.

Back then, and coming from heavy Linux usage for five years, I could have commented on MANY, MANY things that could have been done differently. A sort of “wake up, there is an entire world outside of Microsoft” call.

Windows 8 though – blew my mind. On more fronts than one. They seem to have picked up literally every piece of goodness I loved about all the companies/products I’ve admired and made one big culture/product/direction change – the development models, the quality bars, the communication and announcement model, the feedback model, the structure and discipline, the humility, the confidence, and the list is endless.

On the first count, the team is structured – intensely focussed, small and targetted teams, determined to win, and dedicated to a quality bar (as Sinofsky commented in his keynote a few minutes ago, much to my pleasure, “we are bound by quality, not time.”) Everyone knows what they are doing, and why they are doing it, and how it fits in for the customer. The team is also, unlike many other (or perhaps older) MSFT orgs, not looking for technology resume-fillers. Today’s keynote was about what the whole “thing” can do instead of how cool their new memory-management system is. Windows Live was about how your photos are accessible instead of how their data-access APIs are serialized on-the-wire. Plus, demo’s, not hypothetical scenarios for some  feature nobody else will have in 2016. It’s out there. It’s now. You can download it yourself in less than 10 hours. Now that’s something that I can feel proud of, and brag about as a fellow employee. The best way to respond to a critic is to give them a URL, not predictions or prophecies.

On the other hand, unlike a certain other structured company out there, the team can’t be accused of not being accessible to you either. Developer preview goes out tonight. Feedback is welcome, and will be listened to, acted upon. While the above reminds you of Apple, this part distinctly reeks of the open development models out there. I think this is the perfect balance between the Cathedral and a Bazaar. A purely democratic Bazaar leads to little polish. A Cathedral doesn’t scale. There are multiple official blogs where the teams are listening for feedback. Discussions that criticize the product are not banned or hushed-up.

However, and this is the cool part, if you do want to know about the actual APIs and the on-the-wire formats, it is coming, and it shall be given too. I’m just very excited how the scenario was the primary driver. In my opinion, when I think about whether or not I want to go on a certain platform, my first instinct is “who the heck will use this?” While a lot of people will disagree with me, and if you do, you may be making a big mistake, developers don’t choose a platform because it provides dynamic typing and annotations, or XML-defined UIs. Developers use platforms because: a) Their customers will use it and b) They can delight their customers with what they can build even if it needs to be written in Assembly Language. They will build runtimes or platforms to ease their life, sure. To steal and paraphrase a quote from Jurassic Park: “You can’t contain developers. Developers… will find a way!”

Just because I hate maintaining hyperlinks, you can figure out everything here: http://www.buildwindows.com/

I’ve spoken to plenty of people all over the company, and I loved the fact that Win seems to be the least obnoxious (if at all) of the teams out there. If you have opinions, they want to know. If you have issues, you will get guidance. If there is a decision, you will hear a justification of why they came to a certain conclusion, and if you can show a flaw in their logic, are open to reconsidering that conclusion. If you have nothing but praise, they will humbly thank you, and tell you that there is more that needs to be done, and get back to their coding. :-) That’s one admirable team, and I am proud to work beside them.

September 1, 2011

On code reuse and maintainability

Filed under: Science, Technology — Tags: , , , — archisgore @ 12:32 am

A wise man once said, “Any procedural program, given a sufficient level of complexity, will end up implementing some form of Lisp.” (If you can’t find a paraphrase of that quote, then attribute it to me – but I’m pretty sure I read it somewhere 10 years ago.)

Today, we continue on the rant against frameworks, and look into code reuse and maintainability. Unlike regular posts, this is one area I’m not too sure about, and would love comments or counter-examples. Spare no punches!

What is reuse exactly? Syntax or semantics?

Let’s start with an example. Given a procedural language, I write a framework to do the following. Instead of the coding having to decide at coding-time what operation they want to do on a float, they can delegate it to my run-time operation-definition framework that allows data-driven dynamically loaded operations.
1. The code itself will look like this:
y = f(x)
2. The executable will add a configuration file for my framework that says:
<define function=”f” definition=”System.Math.Sin”/>

This one raises a lot of questions that I want to ask, but for now, I’d like to know, would you call this code reuse? I can certainly make great arguments for this style of coding (keep quiet functional-programmers, this one’s for someone who _has_ chosen a procedural language.) It allows me to change the definition of ‘f’ at runtime. I don’t have to worry if tomorrow my computation changes, because a simple config-change will make my code work for Cos, or Tan or whatever else the user needs. I can replace the definition of “Sin” to use a different implementation whenever I choose.

For me personally, this is bullshit! It’s the worst kind of code I would ever had the displeasure of dealing with. Only the last argument made any sense, and there are ways around that. It is the most irresponsible style of coding too – the programmer, instead of taking responsibility for ensuring correctness of code, delegates every function out of their own scope. If you really want to do that, use a functional language already! People have been advocating them for over four decades now, and this is exactly the reason why! Stop contaminating my procedural code with a smarty-pants half-assed implementation of something for which robust implementations exist already. You can replace your code at runtime and any interpretor worth its two cents has a decent JITer. Semantically, what would be the difference in sending the interpretor a new file to interpret, versus changing configuration for a running program? y=f(x) is certainly not going to have bugs (and can be tested easily.) Your probable bugs are going to be inside ‘f’ anyway. So while your core ‘executable’ can be assured of being stable, it’s a false perception.

The problem with this snippet, functional language or not, is that instead of ensuring correctness, it actually reduces it.

For one, what you see above, is an example of syntactic reuse. You are reusing the syntax for making a function call. You may disagree with me on this, but to me personally, what is really valuable isn’t syntax reuse but semantic reuse. Implementing a good Sine function is damned difficult. That’s what I want to reuse. Calling into the Sine function isn’t what I worry about when I open up my editor. The correctness of my Sine function is what I want to reuse. If there’s a bug, and someone fixes it, I want the new Sine function. If I may ever need to use a Sine function from a different library/implementation, well, seriously – change references to the new library and recompile your code (I know I’m making some atrocious demands here.)

The second problem is the really serious one. When I’m writing code as y = f(x), what the heck am I thinking? I mean seriously. If I am writing a program, I write it for a specific purpose. If I’m computing some vector component across one axis, I know why I’m computing it along that axis. Which means, when I write f(x), I had better damned well know what that ‘f’ should be. If that ‘f’ is ever going to change to ‘g’, then that’s because my problem statement has changed. It alters completely what I am doing (two axes are never the same.) If I start computing Cos(x), it is very very different from computing Sin(x) and I would have serious justifications for why I want the Cosine now. I sure as hell don’t want to reconfigure a running program to do that. I may do a host of things with a running program – use a more accurate Sin implementation, use a faster Sin implementation. If I’m fundamentally changing the definition of the function, I’m in big problems from the outset because I’m changing what my code is guaranteed to do.

Copy-fidelity

I know a lot of people don’t consider the fidelity their code preserves when it is xcopied from one place to another, but I assign it a very high value. The problem with the above snippet, is that 9 out of 10 times, someone’s going to only pick up the code file, without caring what the configuration file is. This is certainly not unreasonable, regardless of how senior or experienced you are. If you see a file called “eigenvalues.java”, you think to yourself, “Hmm…. maybe I should copy eigenvalues.java and use it to compute my eigenvalues.”

I find nothing wrong with this thinking. Very soon though, you see a compiler error: “Function ‘f’ not found.” You spend a couple of days (and if you’re lucky, you’d find a comment) figuring out that you have to host this class file in another loader called ‘function-replacement-framework.jar’. No big deal you say, I love myself some helper tools! This is when you go mad. “functionReplacementParserError: Please define ‘f’ in the configuration file.”

Now you are, in effect, figuring out how to compute eigenvalues so you can define what ‘f’ should be. I’m sorry, but you really should NEVER have to write a config file to define _what_ your code does! Regardless of how much computer-sciency stuff your parser and interpretor are doing, and how they’re generating binary classes at run-time which are processor-optimized by the JITer, this is some pretty bad design there.

Maintainability

So I’ve been thinking about this for a few weeks now, and want opinions on. I came to a good working definition that I think I’m going to use for a while in the near future. I draw this definition from how real-time systems are defined.

A brief overview for the uninitiated – while the common-sense notion of real-time systems is ‘they’re really really fast’, the working definition is ‘time-deterministic’. Meaning that, given what they do, they must do it in deterministic time – meaning, they have to guarantee that something will happen at (‘by’ if you’re soft-realtime) a specific time. Even in light-hearted situations such as filling ketchup in a bottle, you need hard time-determinism, to ensure the bottle is under the nozzle ‘at’ a certain time, not ‘by’ a certain time (if it passes from under it too early, you’ve got a mess to clean up.)

In a similar fashion, I was wondering if maintainability is “doing less work”, or “ensuring correctness even if it is more work.” If I had to make the choice between say…
1. Being able to replace all definitions of “Sin” with “Cos”, using a single config change, which is admittedly a lot less work, but depends on the hope that everyone has taken care to handle the cases where x=0,
2. Or replace Sin with Cos in all the code manually, which is a lot more work, but you ensure that that specific place really is worthy of a “Cos” function as per your replacement-intention. This would guarantee determinism in correctness, but increase work significantly.

If the above sounds unlikely I can certainly come up with more concrete stuff. We frequently use stdout and stderr to send output of a program. Would you prefer to control what goes where implicitly, or would you prefer to modify code yourself (there’s a difference between code consciously having an if-then based on a configuration parameter, vs, your code just having fprintf(outstream, “<stuff>”) where you don’t know during code-time what outstream could be, and can’t function without a complex config file.)

I intentionally chose the stderr vs stdout example because it is vague. I can see it from both ways. In some cases I am conflicted as to whether a warning should go to stderr or stdout. It depends on what kind of tools are going to capture the output, and how they may want to parse/interpret it (some tools, for instance, may consider anything being spit to stderr as indication of program failure.)

How would you define maintainable (or is it maintenable) code? How do you draw the line between abstraction of details vs. core purpose of the tool which is what makes the tool what it is. Does configuration at some point, become complex enough that you’re really just programming in a declarative language through your config files, whereas your “code” is simply an interpretor at that point? If so, is it configuration any longer? If declarative code is more maintainable, why not use a declarative language from the ground up?

August 27, 2010

The Electronic Voting Machine issue in India

Filed under: Politics, Technology — Tags: , , , , , — archisgore @ 5:53 pm

I never looked at my blog as anything more than selfish gratification, until quite recently when a person named Hari Prasad got arrested last week for allegedly having “stolen” an electronic voting machine.

First some background – ever since the EVMs were used in elections, my mom has been involved with a group of politically-un-allied activists. Naturally I made quite a bit of fun of her (my family always enjoys a bit of a jest at each others’ expense.) She used to visit me in Hyderabad often on account of her meetings with Mr. Hari Prasad who has his offices in Madhapur. She introduced me to him on multiple occasions but I always took the meetings casually, being more involved in my “work or whatever.”

You may imagine my surprise when one morning I wake up and see this same Hari Prasad an internet sensation making the headlines on Digg and Slashdot. Then you would have seen me telling everyone, “I know that guy personally and I know that he knows what he’s talking about.” I just found out my mom is in Mumbai awaiting his release and has been subpoena’d (not sure by which side right now), and decided to at least bring the issue to attention. Let me be honest, an year ago, she was in Hyderabad at least five or six times, and while I did believe what she said about the machines, I would never have imagined that they would be taken seriously, and let me be the first to say, I am sooooooo happy I was wayyyyy wrong! If you met those people, they’re really just electronicians – these guys aren’t politicians, and they don’t know squat about that stuff. They know how chipsets work and serial ports work, and that’s all they are making claims about.

To reach slashdot and get that much international attention, to get arrested is pretty impressive. What’s more, I called a few friends and family members in India right now, and nobody down there has any clue that this is even happening. That was a bit disturbing frankly.

So what really is the deal with the voting machines? Quite a lot really – I’ve heard discussions and arguments right from having found the seals broken on the boxes in which they were being carried, and the fact that the storage chips on which the numbers are stored could be plugged out and replaced with relative ease – and this stuff is what they teach in Electronics 101.

I don’t have all the specs with me right now, but I’ve been talking to these people enough where it warrants at least some looking-into by voters before you make up your minds. Whether you are for the winning party or not, as Perry Mason would say, everyone is entitled to a defense because it protects us from being falsely accused of a crime. In the same way, even if you love the winning party, it is in your best interests to at least give attention to the matter so that you are protected, should the system be compromised against you.

The real issue from a common-sense point of view that every really seems to overlook is this: that the “count” stored on the machines is virtual. You see, everyone makes comparisons to conventional ballot-boxes, and a casual “what’s the difference really?” kind of arguments. What they don’t realise is, in the old system there were physically 10,000 pieces of paper that necessitated tampering. An interested political party just needed to hire a street-side loafer to follow the van that carries the sealed ballot boxes from the voting booths back to the election-commission offices to see that nobody brought in another set of a thousand or so pieces of paper to replace the originals. Then again, when the boxes are sealed/unsealed, there are witnesses who sign the locks. Even the ballot papers used for counting can be verified for authenticity and their authenticity can be questioned (if you notice an inkjet printout, it’s a no-brainer.) In short, the system has multiple checks in place to ensure lack of tampering.

In an EVM though, all these checks and balances go out and what we get is: Party A: 5000, Party B 20,000. These are pure numbers. There is no public-key system that ensures even 25,000 different people walked into the booth. There is no way to “go back”, or trace tampering. There is no log of when entries were made – even a text file that contains time-stamps of actions without any Personally Identifiable Information (PII) would make tampering that much harder since a scammer would have to fabricate a large text-file and make sure it’s consistent. Heck, someone could even look at what sectors/clusters each of the block of the text-file was stored in to provide an indication whether it was generated over a period of 5 hours, or was just copied from one large blob.

Does it make forgeries impossible? Of course not, and those claims existed against the old-school ballots too. But does the current machine make forging ridiculiously simple? Yes. For anyone politically inclined, I would encourage you to at least check out his youtube videos. I’ll provide more edits to this post with more details on where to find information.

August 17, 2010

What does 100% CPU usage mean?

Filed under: Technology — Tags: , , , — archisgore @ 1:32 am

Traditional scientists (mathematicians, physicists, and engineers in traditional disciplines of mechanics, dynamics, etc.) have always known that certain entities are temporal in nature. Temporal means time-dependent (using the term losely here), meaning quantities or values that only make sense “across time”. The larger time period you observe them for, the more sense they make, and the less you know about local details (you lose details of when  happened but you get a greater understanding of what exactly happened.) This is Heisenberg’s principle applied to time-domain metrics. I don’t want to go into all the stuff Fourier gave us, but heck that dude really changed the way we do stuff today.

A quick refresher example (or introductory example for those who didn’t study a lot of signal analysis). Imagine you were sitting in a large theatre at 8:00pm in the evening with the theatre partially filled up (say 30% of the seats were filled when you entered at 8:00pm sharp). At 8:01, you observe one person coming into the room. Given this data, would you be able to describe either, by what time the theatre would be filled up, or how many people would be in the theatre at 8:30pm the same evening? Now you doze off for 3 minutes, and again observe at 8:05, you notice another person entering the room (due to the darkness, you don’t know how many people are in the theatre currently). A better estimate of the answers to the two questions? Suppose the first person who entered at 8:01 clarified that between him entering and the next person entering at 8:05, he can assure you nobody else entered while you were dozing off, would that make your answer more accurate?  If by 8:20, you got details of how many people entered when, would your estimate be even more accurate?

As you see, when making such estimates, specific point-data has very little value. Given, at 8:15, there were 40 people seated, is quite pointless to figure out when your show should start. Without knowing how many people are currently seated, but the rate-of-entry per minute is much more valuable. Knowing both, is even more so. Knowing the rate-of-entry as it differs each minute is even more valuable (whether it is decreasing, increasing or constant.)

This is all fairly introductory textbook stuff for most other disciplines. In computing though, a lot many programmers while aware of temporal quantities, either misinterpret them or overlook their usefullness. This can mean a lot of implications in terms of quality, performance and correctness. I comment based on some observations and interpretations I have seen in this industry over the last few years and how we misinterpret benchmarks and metrics.

The most commonly misinterpreted statistic thrown around out there is “CPU usage”. We see people panic at “100%” CPU usage, while there are also apps out there who could have only 10% CPU usage but bring systems to a crawl. Ask a common coder, “My app uses 100% CPU sometimes, is that good?” and the immediate response is, “What kind of coder would write such a program?” Let’s look at how a CPU works for a minute.

The CPU runs on a wave (a square wave to be precise). It has a certain number of vibrations per second, and it does work every time it vibrates (just like the piston of an engine). What you see as CPU usage for a certain program, is the number of vibrations of the CPU the program used to do its own work (basically if a car engine could divide it’s piston strokes between the axle and the air conditioner’s compressor, the amount of strokes it allocated to the axle). A lot of times, we have a tendency to interpret “100%” CPU usage as bad. While this is generally the right metric to use in very generic terms, when you are developing a controlled system, it could lead to some quirky scenarios.

A CPU, just like the car engine, is running whether or not it is used to get work done. Of course, 100% usage naturally means nobody else gets a bit of that power when they need it, but there’s no reason why it shouldn’t be used whenever possible (when nobody else needs it). At times, you turn off your airconditioner, so that you get a higher boost in acceleration. In the same way, for certain applications, if a program is deliberately NOT using 100% CPU, it is a very very bad thing.

I’ve been developing server applications for a while now, and this gets brought up a lot. When my webserver is hit with one request, the server goes to 100% CPU usage for about 2-3 milliseconds. This isn’t only not bad, but actually helpful because what else did I expect the server to be doing anyway? What would it mean if I got 20 requests in one second? The server would still use 100% CPU and answer the requests in order and they get answered faily fast too. I’ve seen people going nuts on forums when they see their OS running at 90% CPU – you can imagine how the question is framed, “If with only a one request it used 100% CPU, how will it handle 2 requests at all?”

It’s not all that hard to reproduce on your home desktop too. Ever notice how media players always seem to be using 80% CPU and the system is still responsive (I compile large code-trees while playing a movie on my 2nd screen)? Well, why shouldn’t they, if nobody is using those ‘piston strokes’ for driving the axle? Contrary to that, sometimes a program goes unresponsive, and you open up task manager but you see barely 1-2% of total CPU usage, and wonder why the program is stuck? Happens to me too – even on programs I’ve written, until I realise that the whole “noble coding” era is passed. Back then we used to use more “efficient” workarounds to common functions to do things faster. Modern OSes expect you to be more semantic than syntactic – tell the OS what you need in no indirect terms, and the OS knows best how to provide it to you. Try doing custom memory tricks, and you end up with inefficient code. This doesn’t mean that the “hacker culture” doesn’t exist where super-smart minds exploit new ways to improve speed and efficiency, however it’s just that no longer can you read books on using “a+a” instead of “a*2″ and hope to gain a lot of applause.

Theme: Silver is the New Black. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 122 other followers