Lessons we learn, lessons we teach

Lately I've been a little involved in both (learning and teaching that is), and wanted to share some thoughts on the subject.

Do you like science fiction? I used to like it (when I was younger, so much younger than today). Asimov, Bradbury and such. It's especially fun when the story mixes past, future and present; reality and imagination. What fascinates me most in these stories, is how the author is making deep and painful observations about reality through moving the characters into a completely unrealistic setting - future comes handy, because it is hardest to imagine . It's just amazing how, by disguising things beyond recognition, the author breaks the thought conventions and emotional associations built in our brain, and doing that lets us understand the reality beyond what we were able to before. John Lennon said "nothing you can see that isn't shown", right, but you can imagine what you don't see, and that's how you "know the unknown" a little bit better.

There's this recurring theme in sci-fi stories "smart man builds machine, machine becomes smarter than man, machine rules man". Is it a real threat? If so, how can we prevent it? Should we stop building smart machines? Is high-tech going to destroy mankind? I don't think so. For one, if I did think so, I would quit and become an organic farmer, and I haven't done so... so far. But the question does bother me. And the answer is, I think, that if we want to continue building smarter machines, we absolutely need to keep building smarter people, or we'll end up like in those sci-fi stories.

The main purpose of education is often perceived as passing on what we already know to the next generation, so they don't waste time on rediscovering it. But it's only secondary. The real goal should be passing on the ability to explore the unknown and solve what previous generation haven't solved. Because if students can think at least as well as their teachers, rediscovering something will be just a small detour for them on the road to Knowledge. But without a developed mind they have no legs to walk that road. And it's a steep road up, so when they stop, they (inadvertently perhaps, but) inevitably slip to the bottom. So how do we nourish the ability to think?

It's not what we know, it is how we learned it.

When asking ourselves how to teach, we should first turn to introspection - how did we learn? If the way we've been taught made us discover all those great things that we are so anxious to pass on, why don't we teach what we've been taught? Sure, we will throw in a bit or two of what we've discovered, but basically why don't we build from the same grounds? They say it about parenting - if you like the way you were raised, you'll likely be a good parent, because you'll repeat what you saw. However, if you look around in what's going on in education system, in almost every level, you see the curriculum constantly changing, "updating", "modernizing" etc. "It was true then, it's wrong now. We don't need it." Why? Because we have the technology? Big mistake. Ancient Greeks, Hebrews, Egyptians, not having the technology, weren't even tiny bit stupider than us. Without the wheel, there would be no Internet, speaking of which, who invented the Net? But no, we don't need old men, old books and such, religion, history... we have TV commercials to teach us how to live, we have the technology. And once technology costs money, suppliers of technology don't want us thinking independently, because, Google forbid, we may decide we don't need it! In computer science, this is the nightmare sci-fi writers were warning us against, and we should at least try to prevent it from happening.

We can't go forward without understanding the past. Students can't possibly understand Java and Object Orientation before they understand procedural programming, functions, math, logic. Then, when (and if) we show them objects, let's show how they came about, and not a popular imitation. If we have a great book and curriculum that generations of computer scientists and engineers grew on, why are we throwing it away? Let Java, Python and their patrons wait. They will lay their heavy paws on the students in just very few years anyway and will turn them in Dilberts, Wallies, and Alices, converting large XML files to long stack traces. Let's give freedom of thought and curiousity a chance to grow just a little bit in students' minds, so at least some of it can survive through corporate development.

It's not what a technology does, it is how and why it works

We're all the time obsessively looking for solutions to our problems. We barely stop and analyze them, until the solution itself becomes our biggest problem. I watched this QCon presentation a while back, and it was a deja vu in many senses. I encountered problems like that, and I even solved them in somewhat similar way. I may be wrong, but what I get from the presentation on the technical level, is that sometimes Object Orientation as we know it (C#, Java), with all the patterns and practices and such, does not solve our problem. The problem in the presentation reminds me of the expression problem - data and operation-set need to evolve, how do we express the relationships? The proposed solution (although I may be getting it wrong) is to have interface per operation; then the implementation of the interface, using reified generic type parameter, stores the type of object it applies to; then at start-up something wires together data types and implementations; then it all becomes a big happy family of multi-methods, operation implementation chosen via dynamic dispatch on the data type. And an old saying goes "when the problem is hard enough, you will find yourself re-inventing Lisp to solve it". It's interesting how Udi describes arriving at this design - a team of people were struggling with the problem for years, until an "old programmer" came to the project retrospective meeting and said "make your roles explicit". Udi took the guy aside and made him explain. Then Udi (and his team, I suppose) implemented it, and now reported the success at QCon. Who was that "old geezer"? What was he? Udi didn't say, in the presentation he is portrayed as a little green troll. Anyway, why do I care who that mysterious character was? Because I didn't come to computer science for some wise Merlin to tell me where the Holy Grail is, I want to be that Merlin! Unfortunately, I am probably not smart enough, but someone else is. That's why I think we need more wanna-be Merlins; wanna-be Kings we already have plenty.

Since I became interested in functional programming, for example, my Java coding style changed significantly. I thought I knew how to use Generics, only after a taste of OCaml and Benjamin Pierce's writings I realized how little I knew about types. I thought I knew how to write object-oriented programs, only after playing with Smalltalk I realized what object oriented really meant. I may not be able to use Haskell for the day job, but the concepts of immutability, closures, function composition, laziness, are helpful no matter what language I use. Let's look at one example - the (in)famous problem of object-oriented design: does square extend rectangle, or in other words does circle extend ellipse? In both cases the former has 2 distinct properties (edge sizes or focal points) and the latter has only 1. Bob Martin discusses the problem in his article "Design Principles and Design Patterns". Does he offer a solution? No - "design by contract" and Eiffel and some hand-waving. On the other hand, understanding types helps, because then I can "design for subsumption": I ask myself - if Ellipse is a type that describes all ellipses in the world, do they all have 2 distinct focal points? No. Then maybe I shouldn't have methods to get and set them. Maybe my API should try to follow the definition of ellipse more precisely. That surely helps to design good APIs. Furthermore, magically, once we turn the shapes to immutable, the problem almost goes away. If Rectangle has a method Rectangle transform(x,y) that produces a new Rectangle with given sizes, then Square can inherit it with no problem, it would simply produce a Rectangle, not a Square when x != y. Same trick would work for Ellipse. After all, shapes are math definitions, why should they be mutable?! See, a bit of "functional" thinking solved the problem. And the moral - understanding the classic foundation of computer science is necessary for programmers.

Reality distracts clarity of thought.

If there is one more thing that we can learn from these sci-fi stories, it's that detaching things from reality may actually increase our ability to grasp them. However education in recent years is insisting on "examples", or worse - "realistic examples", or even worse "examples of being used in the industry". I am not saying the above is worthless, or unnecessary, but it should not be overrated. At some point education ministry decided to teach elementary school math with actual objects - sticks and such, rather than teaching kids the abstract idea of numbers, and it was a disaster. Math, in general, cannot be taught by following "real world" intuition. Nor logic. Physics has evolved way beyond relatively "intuitive" mechanics. So why are so many educational institutions chasing "real life" technologies, at the expense of classics, and ignoring the "too innovative to be popular"? I know why, of course, - money, pressure from the industry, pressure from students who want real jobs after they graduate. But resisting that pressure is absolutely necessary, for the sake of future generations, to save our civilization! Luckily we still have some universities in Northern Europe :-)

I noticed an interesting phenomena with students - when they are "fresh" and don't carry a baggage of "field experience" (C, Java, curly braces etc.), it is easier for them to take a "different" point of view, to understand more abstract ideas. Generics is one example. Teaching them to experienced Java programmers is extremely hard. But with undergraduates it is, surprisingly, much easier. So why don't we teach Haskell for types, Smalltalk for objects, and maybe C for low-level stuff? Then when they meet Java, or any language they will likely encounter in the industry, it will be a piece of cake to learn. Furthermore, some of them will be able to design the next Java!

The return of forgotten design patterns

Some design patterns are used all the time and their names are known to all - like facade, factories and proxies. Some design patterns are more popular than they should be. But some, although rarely mentioned by their name, have been recently "rediscovered". I'm talking about Flyweight and Memento.

Flyweight
This one basically lets us share n instances between m concurrent clients when m > n. It separates "intrinistic state", the normal class members, and "extrinistic state", which is maintained via parameter passing and return values. Make your intrinistic state immutable, and you can share the same instance between multiple clients. Cool, ha? Priceless. Look at message passing concurrency instead of shared memory concurrency, REST, transactionless architecture... All these treasures actually follow the same spirit as Flyweight.

Memento
This pattern suggests that if you want to save your object state, you better export it in a new dedicated memento object and store the memento. Then restore your object from the memento.

This is like serialization, only serialization didn't follow the pattern, unfortunately.

Josh Bloch suggests we do it manually with so called "Serialization Proxies" - see item 78 in chapter 11 of 2nd edition of "Effective Java". Here's a slide from JavaOne 2006 preso:

The book lists more advantages of the pattern, like improved security (see item 76 - danger of hackers acquiring references to private fields using de-serialization), ability to de-serialize a different class from the originally serialized instance (RegularEnumSet and JumboEnumSet example) etc.The name "memento" isn't mentioned though.

Now imagine persistence architectures actually using intermediate memento objects... instead of modifying actual objects bytecode, breaking encapsulation with access to their private fields, imposing constraints like public empty constructors and so on... Maybe we would have been better off with mementos...?

Static types are from Mars, Dynamic types are from Venus

Venus and Mars

static - associated with logic and acting by the rules, strong, efficient, usually responsible for safety and order enforcement; but non-compromising (for better or worse), non-adaptive, weak in communication skills.

dynamic - associated with beauty and elegance, light, good in communication skills, can usually be easily made to do what you want them to, change all the time and adapt to change well; but unpredictable, act on intuition rather than logic, often seen as less efficient and weaker.

---

Types and components

Just some thoughts following a recent conversation I had. Don't we always want static types? If we can detect errors in our program, why not do it as early as possible? Sure. So when is "as early as possible"? I think the answer depends on what our program is - is it one monolithic piece or a component?

In the past monolithic software was prevalent, but today I would bet that most software is meant to be a component. Just look at open-source as it used to be before Java - you'd usually download the C source-code, change it if you need, build it all on your machine, and create one big executable. (And BTW - congratulations, you are a geek.) Cross platform Java changed the situation. Now open-source projects give you a jar to download, you put it on your classpath and you are ready to go. This lowered the bar for open-source adoption, and contributed (aside less restrictive licenses and other factors) to baby-boom of open-source frameworks and tools. Nowadays a Java developer cannot even imagine having no 3rd party jars in the classpath!

Static type check validates our software component against other components in the compilation environment. Does it match the runtime environment? What about different configurations of the runtime environment - there are tests and real deployments, and various types of deployments, and any given installation environment can change over time - new components being added, other updated or removed? How do we guarantee that compile time checks still hold? The short answer is - we can't. We need dynamic type safety anyway. Now let's examine the added value and the price (yes, there is one!) of deeply static types.

On code organization level, we try to reduce the dependencies to bare minimum - hide classes behind interfaces that we hope will remain more stable. The problem is that number of interfaces and factories in our application grows, while pursuing modularity we sacrifice simplicity... So maybe the problem is in the name? Some go as far as add support for structural types, minimizing dependency to a single field/method signature (not a problem-free solution, but there are interesting refinements). All this may help, but doesn't really solve the problem.

Another aspect we need to deal with is building and packaging the software. Here we enter the world of dependency management, the world of "make", Ant, Maven, repositories, jar versions; if it's a large enough and complex enough software we work on, simply speaking - we enter the world of pain. I still find it strange that we haven't found a better way.

As for application deployment and its problems, we'll get back to it later. But the truth is that no matter how hard we try, we can't guarantee there will be no errors when we deploy our software, so ... JVM doesn't trust us and gives us verification.

Simply put, when class is compiled, some of its requirements from other classes are captured and encoded into the bytecode. Then JVM would check them when the class is loaded, and reject the class if they can't be met. (This is really an over-simplified description of a complex algorithm, which also takes time to execute, despite optimization efforts on JVM side.) So this isn't really a dynamic check, it's something in between - names in our class get linked when it is loaded. In the classic Java SE class-loading scheme, where components are basically a chain, this scheme should work. But if we want real components, ones we can add, override, replace or remove while program is running - sweet turns sour. Our interfaces and factories have names, and classes that represent them need to reside in some "common vocabulary" usually loaded by the parent classloader, because it's not only the class bytecode that matters, but also who loaded what. Since we are talking actual classes, not their names, once we loaded two components, they cannot change their protocol of communication without reloading their parent, they also can't use a different version of a sub-component that the parent component has referenced.

In JEE that sort of things is necessary, that's why classloading in JEE is a terrible mess, not only it does not follow any specification, but it is different in almost each and every app server (wasn't there supposed to be portability?!) If you ever used commons-logging in a JEE app, you probably know what I mean. Maybe it got fixed lately, I don't know, but the Tech Guide for commons-logging is an ode to classloader frustration.

Back to deployment: whenever there are some sort of dynamic components - JEE, Spring or OSGi, there is always reflection. And most of the time there's lots of XML too. It's an escape route from static types. I attended Alef Arendsen's session at JavaEdge that presented OSGi and SpringSource. I carefully watched Alef juggle between XML, source and console like a child who watches a circus magician trying to uncover his tricks. But I didn't quite figure out the magic. And that was a whole session just for HelloWorld. I know Spring folks are doing best they can, and they're smart and all... but comparing to Smalltalk, I wasn't quite impressed. As for other solutions, although I haven't tried this out, there's Guice/OSGi integration without XML and with dynamic proxies and on-the-fly bytecode generation with ASM, but there's some overhead for the user, because it requires intermediate objects for services. So this way or the other, looks like JVM platform is holding us back.

Verification is addition, not replacement of dynamic checks. So what we get is basically a triple check of correctness (javac, verifier, dynamic) but loss of flexibility - we are interfering with components runtime life-cycles. If the invocation target is resolved just in time when the call is made, nothing precludes the target component from being reloaded between calls. But with preemptive validation, we get a static dependency tree at runtime, classes wired with each other "too early" and for good, which makes reloading a component very hard (although people keep trying). The reason for "early linking" is also performance, but late binding doesn't mean that the runtime platform can't do any optimization heuristics... but they'll have to be dynamic optimizations in the style of JIT. Will invokedynamic bring the salvation?

It seems when we are talking about multiple components, "statically typed platform" does not quite do the job. Static type check may mean a lot inside a component, but as for inter-component communication they are not only useless, but harmful. People sometimes dismiss dynamic types, because they think "it's like static types, but without static types". What they may not realize is that you are not just loosing, you are gaining something with dynamic types. You get late binding and meta-programming, and in a multi-component environment, it means a whole lot!

And that's when we are talking "inside the platform" components developed in the same language. Once you work with a system that runs on a different platform or developed in a different language - our type system doesn't normally stretch across the communication boundary. The other system may not even have static types, and since we are only as strong as the weakest link, our static types don't really help us. I think every time we try to encode types into communication between systems we end up with a monster like CORBA or Web Services. But there's another (unfortunately popular) extreme of just sending a string over and hoping for the best - with no checking on our side at all. Then we are relying on the other system to stop us from doing damage, and there's no way to correctly blame the component that made an error - was it a wrong string or an unexpected change on the other side? I think that ideally type or contract checking and conversions can be done dynamically on both sides, and not as part of the protocol. This results in light and flexible data-exchanging protocols (like HTTP or ATOM) which are easier to work with and I think will win in the end. On the more theoretical level I like this model for intercommunication and of course there are Aliens, that model external system as a special object in our system.

So as far as I see - components simply require a dynamic environment, they may be statically checked inside, but act as "dynamic" to the outside world. Sort of hard skeleton and soft shell. Indeed soft parts are much easier to fit together and less breakable, due to flexibility - this is used often in mechanical engineering and in nature, so why not in software?

Brains, bucks and programming languages

The title is supposed to be a paraphrase of "sex, drugs and rock'n'roll" in a geeky context.

This autumn I went to see Paul McCartney in concert - a lifetime dream come true. For most people Paul McCartney is first of all an ex-Beatle. Indeed, during the concert he played many classic Beatles tunes to please the audience. And the audience was very pleased. Then, he cashed in the multi-million-dollar cheque and went back to England to do what he really likes - which at this point seems to be composing experimental electronic music.  To me it looks pretty fair. 

Recently I listenned  to James Gosling's keynote at the JVM Language Summit. I actually enjoyed the presentation very much. One of the things he said, was something like "My dream would be to implement Fortran over JVM ... ah, but I have a day-job".  Now, not that JVM really needs a Fortran IMO. But think about it for a second. How many people in the world can design a programming language? How many of them can design a good programming language? And a popular one? Java is more popular than Beatles. Uhm, well... even if it's not, you get the idea.  Now what can be more important for James Gosling to do during his day job than design a programming language of his choice, I should ask his employer? What? Throwing T-shirts at JavaOne attendants? No, really. Why is it that James Gosling can't do anything he freaking likes for the rest of his life?

I think something in our business is unfair. I am not saying Microsoft model is right, I am very pro open-source and free software and all that. But I'm confused - something about it isn't right. Large IT companies make loads of money, and waste a lot of it on complete crap - I've seen this from inside. So how come Gilad Bracha cannot find funding for Newspeak development? This is totally surreal!

There goes another angry post.

 

DSL - fuel for life

Do you feel that your programming language is too bloated? I do, and I know I am not alone. 

Let's take a look at Java. You may ask - what do you mean, when you say Java? Ah, good question. There is Java, the language, as described in the spec. Java the Standard. Don't you wonder where's the new edition of the book, BTW? Anyway, then there's mini-edition, enterprise edition, real-time Java... there's a huge stack of official/certified technologies (e.g. all the JEE stuff - is JSP or JSF Java? is JPQL?), for which there are often multiple vendors. That's not all, there are all the popular open-source frameworks that don't bother getting Sun's approval, and yet they possess lion-share of the market (e.g. Eclipse, Spring). There's no chance to even keep track of all this, forget mastering. And yet, I don't feel that I have all I need. I have all that, and yet I can't write a descent program the way I'd like to, because I am missing some core features. What? Let's see - how about proper modularity, closures, tuples, local type inference, properties... 

Java made the grade in expanding layer by layer - to the extent where it reminds me the state of the Earth in Wall-E. If you haven't seen the movie, I'll just say that the Earth drowned in human-produced garbage, and the garbage made it inhabitable for anything organic. The garbage in Java makes it impossible for the core, organic features of the language to grow. In the movie, people are leaving, and robots stay back to clean up - now you make the analogy. 

As a Java programmer I really identify with Wall-E. Moving tons of garbage around, day after day, in a desperate attempt to clean up the world - something obviously impossible. And man, I'd love to be that flying-iPod-looking Eve from outer space. She's so strong, so modern, so clean and shiny...  And she has a mission!

If Wall-E is my Java, then my Eve is no doubt Newspeak. 

So what does it mean in a wider sense of programming languages? I think good language should have a small core, as little redundancy as possible. And it should grow from then on. Here's Guy Steel at OOPSLA '98:

I think that the best way of growing is via internal domain specific languages, DSLs, - you don't change the core language, yet you cover more and more domains. Anders Hejlsberg also talked about it at the recent PDC conference - adding features as a DSL; and then, if the DSL is very successful and popular, add syntactic sugar to the core language, as they did with LINQ. I don't think that you need that latter part if your language is well suited for DSLs to begin with. So the language should have a small core and be well suited for DSLs. Java isn't DSL-friendly. Scala is much better, e.g. the Actors library. Haskell, Ruby and of course Smalltalk are really good at it. 

The last part is getting rid of garbage as you grow, and again, Gilad has an idea how to do that. 

I could go on forever and ever about DSLs, showing examples like parser combinators, unit-test and SQL libraries and so on. Martin Fowler is writing a book. So, instead of boring you with repeating what's been said many times, and enumerating lots of references, I will just finish with this cutest quote:

DSLs are for making languages bear-able.

For I am a bear of very little brain and long words confuse me. [Milne 1926]

The premise of this subject is that computers should adapt to the ways of people, and not the other way around.  

The Great Divide

What hasn't been said about static vs. dynamic types in programming languages? Read on at your own risk, because here I go again...

When you think of static types what comes to mind? Haskell? OCaml? Scala? Dear friend, you are better than most of us, but you have clicked the wrong URL. Peace. See you in another post...

Did you say Java? Still with me? Good. Listen, now when the others have gone, just between you and me, the guys from the previous paragraph - they're on to some good stuff. Check it out, you won't regret it. But don't quit your day job, not just yet. It's a bit complicated, but did you ever witness extreme programming methodology implemented in a big corporation? No? Then picture this: Elbonians take over Dilbert's firm and make everybody do XP. They even send pointy-haired boss to a Certified Scrum Master course. Get the outcome? It can only end like the implementation of Carl Marx's ideas in Russian countryside. I am trying to say - there are ideals, and there is reality. In reality, Haskell programs have bugs too.

So Java, you say. How do you feel about dynamic types? Cool? Get out of here. No really, it's no fun preaching to the converted. See you!

Oh no, heaven forbid, you won't touch them with a stick. You're my guy then. So let's rewind to Java 1.4 days, after all many Java developers still use 1.4 and many others look back at it with nostalgia. Are you one of them? Ok. So what about pre-generics collections, do you think they are statically typed? Hmmm... And what percentage of your code involves collections? So this code was not entirely statically checked. Now add all the reflection stuff...

But then of course came Generics. And suddenly Java is much more complex. How do I make my code compile, gee, wildcards, captures... ?! I am trying to get something done, hello!... It's easy of course to blame Generics implementation, but if we learn something from the folks whom I kindly asked to leave in the beginning, they'll tell you that finding correct static type for every element in your program is hard. They of course think that hard is good, they are noble men with ideals, they like overcoming challenges. But you and I, we're just trying to make a living. So we curse Sun and back off to an untyped collection. Hm, maybe we're just doing the right thing? Maybe sometimes we just know that our program is correct, but the compiler demands more and more typing and wastes our time?

Java 5 was all about improving type-checking. If pre-defined types were not enough, annotations came handy. Define your own and test it at compile-time or at run-time. Did it ever happen to you that there were so many annotations, that you couldn't see the code?

See, more types is not always a good thing. Unless you're very keen on intellectual challenges. James Gosling said this about Scala - functional programs will make your brain hurt, they are for calculus lovers. He's right. It's the kind of pain you feel in your muscles when you start working out, you know, that indicates they are still alive... So working out is good, but we can't afford doing it all day, ha?

Maybe the appeal of plain old Java was that it's a combination of static and dynamic checks? So it's not that all dynamic is evil, maybe it's a matter of how much and where?

Give dynamic types a break. Who knows, you may find eventually that they're good for some things.

Peace.

P.S. I've done some role playing here, just for the record. I do love Generics, even though they were hard to master. Annotations are overall very useful. Right now I don't do as much Java as I used to, and I do other fascinating languages (static and dynamic) as I, for long time, wanted to.

Software: Live and Let Die

This week I was lucky to attend Gilad Bracha's guest lecture on Networked Serviced Programming at the Hebrew University. He has been talking about Service Objects for some time now, but nothing compares to hearing it live - Gilad's presentation was witty and fun!

So here is my interpretation and some take-aways.

What's the problem with software? It is too damn complex. Projects crumble under their own weight. It happens to successful projects - our dearly loved Java, for example. Also look at Vista, if you dare. And it's certainly true for the monster-size projects I used to work on. Once upon a time our team took the corporate "quality improvement" policy seriously and decided to investigate what causes bugs in our multi-million lines of code project. We collected all sorts of statistics and ran all possible metrics (which was tricky 'cause some of the tools would choke on such a huge code-base) but long story short our finding was this: the only metric that correlated clearly with defectiveness was LoC. It is hardly news, but a cure to the disease has yet to be found.

There are certainly several things to be done, but what this talk focused on is getting rid of code which shouldn't be there, or in other words - dumping unused code and backwards compatibility. The way we work today - we are bound to not just specs and APIs, but to all the accidental behaviors and bugs in the previous version of our code. It seems that Gilad views code as if it was a live organism. Staying alive means being connected (and network plays a central role in his vision), but we should make way for evolution and some code should die - Gilad calls it "bit rot".

So how do we turn software into a healthy living organism? According to Gilad, there are several things to be done on the technical front first.

• Take advantage of the network: maintain a bi-directional connection with the control center - let programs pull upgrades from the net, but also send back operation statistics. This means that programming platform has to be aware of the network, and aware of the fallacies of distributed computing. This is cloud computing utopia: Internet as a platform, browser as an OS, and Javascript as the low-level programming language into which other languages can compile (in a GWT kind a way).
  
• Modularity: it should be possible to extend and replace individual objects without interference to the whole organism. Gilad has a well developed theory about how modularity should be done in a programming language, based on principles of object-orientation using mix-ins, nested classes and inheritance hierarchies; his new language, Newspeak, is going to implement it.
• Explicit Dependencies between modules - no static, no imports, modules are truely independent and dependency management (wiring) is performed by passing other module instances as parameters to module constructors. This allows to maintain clear boundaries between modules and flexibility in module composition.
  
• Frequent Updates: to allow the "clients" of the object APIs to deal with changes, in addition to maintaining modularity, the changes should be made small and frequent. Call it agility, if you like. That means that we can't afford reboots, and we need to find out when the system is quiescent so that upgrade can be performed, which brings us to the next point...
  
• Reflection and Hot-swapping: objects should allow other objects to find out both static and run-time information about them without breaking the encapsulation. Objects should also allow other objects to modify them "live". Gilad and his team at Cadence are building the support for these features in Newspeak using Mirrors, a concept that originates in Self programming language. It's worth noting that there exist dynamically typed languages that implement hot-swapping today - Erlang being one of them.
  
• Security is important in any distributed system, and even more so if we allow remote objects to mess with the program. So in addition to dynamic typing and pointer safety, Gilad proposes mirrors to be guarded by capability-based security, similar to the one in E programming language.
• Synchronization: many programs need to work with persistent data and it is important to keep the program and the data in-sync. Gilad proposes orthogonal synchronization, based on Smalltalk orthogonal persistence idea, where objects are split into transient and persistent ones by marking object tree roots accordingly. Persistent objects are upgraded whenever the corresponding part of the program is upgraded, and transient objects are lazily recomputed. If the data is ever to outlive the service, it would be exported into some generic format, such as XML.
  
• The most extreme and bold part of this vision is probably No Versions and No Releases - there would be only one version for every program out there. Gilad sees software becoming more of a service than a product, but in order for this to realize we'll have to overcome not just technological, but also psychological and economical barriers - we will have to change the way we develop software and the way we make money of it.
  

Bottom line, all this may sound too futuristic, but "software as a service" and "platform as a service" are making their way in the industry and this wave, if strong and successful enough, may bring the significant change Gilad is predicting.

P.S. As for the rest of us, living in the JVM world, some of the ideas ring a bell. We hope that modularity JSRs and OSGi will improve Java. Those of us who survived Ant and Maven, and felt the weight of a DI framework, will probably appreciate the amount attention Gilad is putting into software composition. It's worth noting the attempts to address hot-swapping on JVM, such as JavaRebel and Jonas Boner's experiments with Scala Actors and Terracotta. Terracotta server also utilizes some ideas which (in my mind at least) look quite similar to the orthogonal synchronization scheme.

Noble cause

Duchess is an on-line community of female Java developers with members from all over the world, but currently active mainly in the Netherlands. I figured they could use some promotion, besides it's a perfect excuse to put this cute mascot on my blog:

Typesy Turvy

Lambda The Ultimate feed just notified me that types are considered harmful. Ah yes, I heard this before, so what's the news? The news is that it's not Stevey vs. Cedric or anything like that, this is Benjamin C. Pierce in his own write. Benjamin C. Pierce, from "Types and Programming Languages" and "Advanced Types and Programming Languages", King of ML, Lord of the Functional Programming Commonwealth, Defender of Type Systems Faith!

Nice presentation, BTW. I think it's one of these situations when a big shot computer scientist is confronted with a real life problem. So you say the language is perfect for writing a compiler - good for you. How about a database application with a web front-end?


(click to see full-size)

Mr. Language Designer, where are you in this picture? Yes you, who designed the mousetrap which "the bug" has safely escaped from. Now it is here, so are you at least by the developer side, handing him something heavy to throw at "the bug", or are you on the bed with the rest of the crowd going "ah ah ah, what do they teach computer science graduates these days..."?

One thing that impressed me during otherwise boring (let me just read you aloud the tutorial) JRuby on Rails preso I recently attended - here is a system that tries to serve the needs of the developer. Not server vendor, not language designer, not JSR politician, not some brandthirsty marketing person or buzz-oriented architect, THE DEVELOPER. I am not used to that. So all I have to say - programmers of the world, unite! Stand for your rights! We deserve better tools, because we are the ones getting the job done.

Thank you for reading.

Girl Power

I have been reading recently that women are abandoning computer science, and that percentage of women in our profession is not just low, but getting lower. I'm not sure actually that the situation here is as bad as in North America, but it's certainly true that software engineering and computer science are not very popular among women. Why? To be honest, I don't have the answer.

Here is what comes to mind:

• Geek-ness is viewed in the society as the opposite of being attractive; this is much more important for young women, than for men. For a guy - a nice high-tech salary will provide the attraction instead.
  
• Sitting in a cube by the computer all day and doing one thing, coding, is probably not very attractive for most people, but for women especially, since most of us are better at performing a variety of tasks and interacting with people - I mean there has to be an explanation why "secretary" or "teacher" are such typical women professions.
• Hi-tech jobs are very demanding, too demanding. For most women family life is at least as important as professional life, and usually family comes first. But there just aren't many positions you can find after getting computer science degree that allow you easily balance work and family life.
  
• Hi-tech is for young people - look around, how many programmers you know are over 45? Why? For the same reason you don't see many women - it's hard to compete with the smart kids when you are pregnant, or haven't slept for a week, or worried about some family matter.
• But can't we just switch roles with the husband? well, go back to first bullet, double standards of the society certainly don't make it easier on us or our partners.
  
• People we are surrounded with (nerdy young men mostly) are pretty anti-social creatures in the first place, even more so with species of the other sex, even more so with the ones that don't fit social stereotype.
  
• Now suppose you survived all the obstacles, because you passionately love science and engineering. Did you watch the knack? There's a grain of truth there... society became less tolerant of weirdness, and people who 100 years ago may have been referred to as crazy geniuses nowadays live "normal life" on prescribed medications. And for a girl it's even stranger to be a crazy genius then for a boy.
• Male domination in the field - yes, it's chicken and egg problem. The field will change only if there will be enough women in it to drive the change from within and help other women. So I will devote the rest of the post to the ones who made it.

Lady Ada (Byron) Lovelace

She was more of a technical writer really, but she was a visionary.

Admiral Grace Hopper


The inventor of COBOL and debugging.

Professor Barbara Liskov


The one from the substitution principle. I chose her among several prominent computer scientist women because she was the first female computer science Ph.D in the US.

SVP Jayshree Ullal


I originally thought to put the Google princess here, but having personally met Jayshree (she was my manager's manager's manager at one point) and impressed by her personality and professionalism (she is so clever, and yet such nice and humble person), I decided she's a better candidate to represent successful women in computer industry.

Alice, Dilbert's workaholic colleague

What did she achieve exactly? Surviving in the office should not be underestimated! So I am going to honor another ex-colleague, although we never met, for providing inspiration for a character I can identify with :-)

Keep coding girls!

My OO My

No, I don't mean the song, but it's to do with Scandinavia. Mads Torgersen mentioned this during interview with Joe Armstrong (highly recommended, there is also a part II), and later I saw another reference in a paper: apparently there is "Scandinavian school of object orientation", sometimes compared to "American school".

Here are some insights on the difference between the two. The point is that the former is about concepts and philosophy, and the latter is about pragmatic aspects like software reuse and organization. I am not gonna pretend that I'm smart - all I know about it comes from searching Google, so you can go there and find out for yourself.

* On a side note, since I am linking to MSDN here, I want to say that even though my last post wasn't very favorable to Microsoft, to be fair - the company plays an important role in renaissance and democratization of functional programming (LINQ, F#, etc.), with guys like Mads Torgersen and Erik Meijer largely responsible.

Open the source, sesame

About a month ago a big shot from Microsoft came to speak at the university - looks like he's on some kind of a tour, and his mission is to convince universities to teach Windows operating system aside Unix/Linux. He waved at the audience with a free license and tried very hard to prove that some aspects of Windows design are better than Unix, mainly because it is "newer" design and better suited for nowadays computers.

Long story short, one of the things he said, was that open source is bad, because engineer who looks at the source of a library will design his application assuming particular implementation, which is now tightly coupled to library internals - bad. Obviously a discussion erupted - design by contract and Eiffel were thrown in the air, relation between programming language and operating systems..., but frankly all I could think of is my distant past as a VB developer and those 1000 pages "Windows *** Unleashed" books. They were written by hackers who used trial-and-error against Windows DLLs trying to make some sense of the APIs - the formal documentation was either sloppy or intentionally incorrect to misguide us, Windows application creators outside Microsoft, since we were potential competitors. Things were so different when I moved to Java. (Though my first Swing experience made me - believe it or not - miss VB, but that's another story.)

Anyway, statement like this coming out of the mouth of a Microsoft employee, even if he has an impressive Unix record, was easy for me to dismiss. But here is a respected Smalltalker saying

"I think people should have the source ...not to get miss quoted saying I'm against open source, but I think it's important class libraries should be viewed like caves."

(Ah, wait, he is from IBM - he can be ignored too... just kidding)

I mean, seriously look at this puzzler for example - it's pretty cool, but the solution heavily relies on a particular implementation, I would even say this is actually a hack. And sometimes a hack is necessary... or is it? Is there or will there ever be a perfect environment where we can program without hacking?

P.S. The reference to Ali Baba tale in the title is intentional. Open source is good, very good, it's a treasure and in the world dominated by large corporations - almost a miracle. But IMO we should resist the temptation to exploit it to the last bit, because we might find ourselves locked in the cave.

Mercurial adventure on Widows

With a friend at work we decided to check out Mercurial as source control system. The repository was set up on a Linux box and zillion lines of code and libraries were sucked in from ClearCase, now I just had to clone it to my lap-top running Windows XP.

Installed Mercurial 0.9.5 via TortoiseHg (I am familiar with Tortoise with SVN so it was a natural choice) - very easy and straightforward so far. Issued clone command - hm, filename too long. Switching to cygwin - doesn't help.

So Ok, we have pretty deep directories structure and Mercurial makes it worse because it translates everything to lower case by "escaping" capital letters - adding underscore before the letter, so "A" becomes "_a" and a single underscore becomes a double one. Why? This is to avoid problems when working with case-insensitive operating systems like Windows. Fair enough, but what do I do with these long filenames now... Python bug? Oh, great. In the meantime the error became "file not found", but I (correctly) suspected the cause is the same.

I found this bug report (and this one) - at least it happenned to other people, it says I can use a 'usestore' workaround. Cool, found this - says change one of the scripts, but hey, they are compiled?! Ok, downloaded ActivePython (I admit, I never touched Python in my life) also downloaded the sources. But apparently I need a C compiler too for the full build from source - that's too much, I decided to just run Mercurial with the interpreter.

Found the right script, changed it. Boom, of course extra space in indentation - until now I only heard jokes about this (besides similar experience with Fortran half a life ago). Nevermind, got it now. Still not working. I take another look at the script - the if statement I just added was already there before! Oops, what's going on? Trying to understand the Python code - it is looking for some 'usestore' configuration option, which BTW is supposed to disable the "escaping". All I need now is to set it. Hm, how? Eventually I find out that I need to edit <TortoiseHg-install-dir>/Mercurial.ini adding

[format]
usestore=False

(If you use plain Mercurial, not Tortoise, create Mercurial.ini under <mercurial-install-dir>)

I do it and... voilà, the repository cloned!!! So bottom line - all I needed was ini file change. It is somewhat dangerous 'coz I still have the mixed case, but at least I have the code on my laptop. Why didn't I find it in an FAQ or tips'n'tricks or something? No idea. Hope this post helps the next person who googles for it.

Listen To What The Man Said

Just couple of nice quotes for today: Alan Kay on Innovation and the not so new but still relevant Edsger W. Dijkstra transcriptions - look at this Q&A for a quick feel.

"Machine capacities now give us room galore for making a mess of it. Opportunities unlimited for fouling things up! Developing the austere intellectual discipline of keeping things sufficiently simple is in this environment a formidable challenge, both technically and educationally."

Modularity - outside in

It is probably a pretty obvious thing, and yet some developers and software architects manage to overlook this.

When refactoring a project under the flag of "better modularity" one needs to start with northbound, southbound and other external interfaces - remove the coupling between the "big boxes" in the system, and only then go inside each box and improve its architecture. Otherwise we'll find ourselves redoing huge amounts of code over and over. Just moving your middle tier code base to JEE does not solve the problem... it is like adding meat balls to spaghetti, while what you really want is lasagna.

Just a thought.