"no fluff just stuff" day two

Posted by anton
on Saturday, May 27, 2006

day two started with neal ford's talk on practical agility. i feel that this is one of the areas i am lacking in - all of my daytime job experience has happened in the environment where very few agile practices took place. i am interested in these cookbook-like recipes that give one a tangible feel for the steps in the process.

no revelations here, but a nice talk reaffirming the facts. his subsequent two-part talk on sunday was far more interesting, actually showing the artifacts produced by the project managers, and telling real-life war stories. it turned out to be more of an introductory talk, but some quotes did stick (or perhaps resurfaced again):

if you are afraid of a piece of code, refactor it
agile methods expose poor developers
maximize the amount of work not done
agile development is a form of risk management

in fact, risk management is probably the best selling point of agile methodology (which neal constantly emphasized).

next was the talk by bruce tate, titled where agile meets argyle: new processes in established companies. sadly, almost nothing there was new to me, or particularly exciting. bruce is not an engaging speaker, compared to justin or venkat, but he is nevertheless a very nice guy to talk to, as long as he is not in the "speaker" mode. i really do enjoy his books and value his opinion, as well as his drive and energy. perhaps i just heard the same arguments over and over again throughout the whole weekend (not to mention reading various books on the subject), so the overall impression was a bit muted. i suppose i really never did stick up to the higher-ups, selling agile, so some of the points seemed somewhat irrelevant to me.

a couple of times i snuck out and dropped by glenn's talk on jvm performance. in retrospect i wish i spent more time there - although most of the subjects on paper looked like something i alerady knew (mostly through personal performance tweaking experience on developement and admin sides, as well as the classic java platform and java performance books), but there were a few points that were news to me (cost of uncontested serialization for instance). talking to him afterward i realized how thorough and how knowledgeable he was when it came down to the innards of java platform.

next talk was justin gehtland's ajax architecture. although i went there mostly for the entertainment value, i still got out a lot from it - just clarifying my own understanding, and, most importantly, learning to talk about this topic, and seeing how well he can engage the audience. him and venkat are complimentary - venkat is a better all-around speaker, but justin comes across a bit livelier, filled with a sort of boyish energy and eagerly stuffing a multitude of other topics into his talk.

talking to neal ford the night before, i was curious to see his "productive programmer" talk (especially since there is a book in the works, scheduled for this fall). i stopped by a couple of times during this talk, and saw all the right things in place - cygwin + unix + command-line tricks, scripting languages for automation, etc. it was a good basic talk, and there was nothing really that i learned (except reminding me the pain i felt when i first switched from norton commander to windows95 gui).

however the topic itself warrants a longer rant, and i am really glad neal brought it up. first of all, i make a difference between a programmer and a developer. i've worked with both kinds (and even extreme examples of both kinds), and for the most part programmers are only useful in a big organization, coding to the tight, verbose spec, perhaps outsourced. a developer is someone that has a toolbox and knows when and how to use the tools at hand. i've seen too many consultants or narrow specialists that do wonders in their respective app development gui/ide, but freak out when it is minimized and they face an OS in front of them, or a task that is done best with something else (just ask me sometimes about the "laptop boy" or "tight guys"). i used to wonder whether the knowledge of the OS and productivity tools is important to a developer, and these days i see it almost as a requirement, better yet - a sign of common sense. a good argument that neal brought up - developers know more about the underlying OS, the way it works, and the way to tweak it to maximize their productivity.

however, there is a difference between GUI/OS productivity tricks (shortcuts, "getting around in a hurry", note-taking apps, etc) and development tools. both are needed, but the former is a bit tricky, since one could use many machines, and tweaking the environment at that point becomes painful (i myself have four machines, and i used to run heavily customized litestep and the likes, but now i just stick to minimal customizations: mostly firefox and cygwin + a small list of IDEs/productivity tools). when it comes down to development tools in one's toolbox, i want to see common sense and knowledge of a wide variety of tools. unix background generally puts people in the right mindselt - given the famous "grep question" from stevey's list of five questions, i want to see a sensible response; and this sort of response comes from a developer well-versed in basic unix tools.

after i switched groups at work about a year ago, i found myself in the environment where all of a sudden i could really help less-technical folks out by a few quick scripts - be it data imports, creating test data, loading some stuff from LDAP, merging a bunch of spreadsheets, etc. it is surprising how much energy is spent on these tasks, only because there is no versatile developer around that could quickly whip up a bunch of scripts. this is when the unix ideology really pays off - a bunch of small tools that read stdin, print to stdout, and could be chained together using pipes in cygwin/unix. this is when the value of a simple for i in `ls *blah`; do stuff; done could not be overestimated. incidentally, someone in my environment that showed a great ability to help the folks out (not just business users, but certified thought-leading architects) was a testing guy - a great example of someone that can build scaffolding around a product if needed, understands why and how stuff works - anything from OS to the end-user product.

my personal acid test for the developer would include a few questions on pragmatic automation - know the scripting languages, know the OS scripting, understand when a 1000+ lines of Java code could be replaced with a few lines of shell scripts. this is when the books like data crunching from pragmatic programmers and "pragmatic automation" come in handy (but a good developer knows all that already, right?).

to conclude this rant, this is why i enjoy the "back to the unix basics" mentality switch that was brought by the rails community. rake/capistrano + linux/mac os x; back to the bare metal and hacking mentality (as in building small, simple things and taking advantage of the basic concepts, but pushing them further).

the last talk of the day was "three technologies to watch" by bruce tate. a very anticipated topic, although he gave away some of it during his previous talks or personal conversations. he started with rails' active record and the way it takes advantage of metaprogramming in ruby. i've already seen most of the talk, since his blog entry on the topic was posted on artima just a couple of months ago. great stuff nevertheless, i wish i've seen it a year ago. next set of slides was on continuations and their applicability in the web servers world. once again, i have already read bruce's article on developerworks. this time it made me acknowledge (once again), how much i am missing because i am not familiar with functional languages (lisp in school and incidental readings on haskell do not count). the idea is great, and i am yet to grasp all the implications of it. a takeway - take a look at continuation-based web frameworks (rife, webwork2, seaside). finally, bruce talked about erlang, yet again exposing my lack of knowledge about parallel systems and the math behind it. instead of my poor interpretation of it, i will suggest reading yet another passionate rant by stevey yegge that talks about parallelism as it applies to CS.

the day was a bit disappointing, since a lot of the talks were based on the blog entries i have read earlier; i guess this is the drawback of being too close to this community and mindset. but if nothing else, these guys are very good speakers, and it helped me to clear up a lot of things in my head which hopefully would allow me to be a bit better at relating this stuff to others.

"no fluff just stuff": day one, the keynote

Posted by anton
on Tuesday, May 23, 2006

the keynote is what finally delivered that "blow your mind" experience that i was looking for. it started rather innocently with the usual run-of-the-mill DSL talk - traditional coffee-ordering DSL, etc. however, a little bit into it, and there was a nice distinction made between internal DSLs and external DSLs - something I have not articulated as clearly to myself.

internal ones are built on top of the language they are written in; they have full access to the underlying language, but they are also bound by its syntax and features. external DSLs is something you write a parser for, and then either interpret or compile them. external DSLs can do anything, at the expense of writing and maintaining a grammar (strangely enough, JavaCC is not used as an example, although yacc and lex are (no bison though!)), and lack of IDE support (auto-completion, color-coding, etc). a very simple way of distinguishing the two, very nice.

next is a statement that OO is not a panacea - not everything can be neatly modelled in hierarchies; not everything is an object. instead a lot of problems are best modelled in a language. here i digress and refer to another very pertinent rant by steve yegge: "execution in the kingdom of nouns." this is a very welcome approach, especially going back to functional languages and DSLs. context is everything; the same functional language can result in different representations, depending on the context (an example with ruby DSL and several different includes at the top that change its behavior).

next is something i have not thought about until now - the fact that none of the IDEs these days work with text - it is all abstract syntax tree (AST) that we work with. this is what allows IDEs to perform refactorings and all kinds of other tricks. this is why IDEs used to require one to compile the code before it could be refactored, but now they are getting better and better at handling broken code (i.e. code where some of the branches are missing). also, do not forget the predecessor of all small languages - Unix shell. it would be interesting to see where it failed, and where microsoft monad is failing.

finally, he whipped out JetBrains MPS - a tool for metaprogramming developed by the creators of IntelliJ. it brings the power of IDE to external DSLs - code completion, color-coding, syntax verification. in other words, one can design their own DSLs, and have an IDE that supports it. the tools looks very promising, although somewhat lacking "productionalization" side. seems like the team is waiting to see what the big boys (microsoft with software factories, and with intentional software) will do, and then offer an product to support it.

this leads nicely to the stuff i have been struggling with lately at work. i do not believe in MDA tools - any generated code is the code one has to maintain, and abstractions there leak very easily, especially when it comes to distributed enterprise code. so the approach to raising the abstraction level and getting closer to the problem domain is to use terse, expressive languages. these languages could be internal or external DSLs, as neal showed in his presentation. alternatively, the same domain (as long as the underlying core representation of the domain is the same AST) could be represented with graphical tools. and this is the stuff that i do at work on a current project - a well-defined space of enterprise integration is modeled using simple GUI tools that include all the building blocks - queues, web services, mappings, flow control, database connectivity, adapters to other apps. i was uneasy at first, feeling uncomfortably close to the MDA world, but now I can view this as yet another form of DSL.

and this leads to another topic i have been pimping left and right: given these simple integration tools and packaged products, a typical enterprise (the one that can be fit into the business model offered by these packaged products and can afford them) will require a very different type of "architect" and "developer". since it takes only a couple of hours to knock out an integration scenario, mapping a message from one system to another, complete with HA/failover/load-balancing/security/etc, then majority of the work would require less and less in-depth technical skills, but more and more of a business knowledge. these people will know a lot more about the business, and a lot less about computer science. granted, the breadth of knowledge is still essential to avoid blunders, and they would be technical people first of all, but a lot of their expertise will shift to the business domain, once the infrastructure and packages are in place. my favorite tabloid happens to reaffirm these sentiments in a recent article: "SOA predicates rise of the enterprise architect".

this kind of architecture still requires a lot of technical knowledge and developer's background, so the promise of "situated software" is still just a dream. it does, however, highlight the difference between the developer and the architect, as well as the different between application architect and integration architect (especially the one that depends on business domain knowledge). my problem is that i still enjoy developing things so much - i like building them, i like seeing them being used by people, i like tweaking them as they run. i like doing hands-on stuff, this is why i still do things on the side; but i also like the challenge of developing large systems that do require a significant domain knowledge. the question is whether my current occupation is indeed a domain that i am interested in knowing more about.

in retrospect, i think a lot of the talk came from martin fowler's article on language workbenches, which follows the same general layout as neal's talk (neal also quotes fowler quite a lot, but then who doesn't? the man deserves it; i just wish i had a better memory). in any case, the talk was excellent, and i am looking forward to digging more into this topic.

the whole keynote is available on neal's site: slides, samples

no fluff just stuff: day first talks

Posted by anton
on Monday, May 22, 2006

this year i was actually anticipating the conference, waiting impatiently for the release from work tedium. i tend to feed on the energy of others, and these intense three-day gatherings are a perfect boost that lasts for months afterward.

this time i chose to attend fewer purely technical talks. i think some of it is the result that i have to work with people a lot more in my new role, and i am trying to figure out how to be better at it - there seems to be a lot of non-tangible things that i am struggling with. another side of it - i am trying to decide what architecture is as opposed to development, where is it that i am trying to go, what is it that i am interested in doing, and what my focus should be, given that the time is limited.

of course some of it also comes from the gratifying masturbatory effect of bitching about work and those cavemen coworkers (the theme that was running strong last year, and something i am too eager to jump into myself).

on friday afternoon i caught the end of ramnivas laddad talk on AOP ("state of aop", to be exact). nothing groundbreaking there, just a nice rundown on load-time weaving (relying on JVMTI agents), bea's take on aspects in jrockit (do not miss reflective calls), xml and annotation syntax for aspects, eclipse AJDT, java5 features support. it was a little disappointing to see the obligatory logging example (i thought any AOP guy is fed up with these). most of this is already familiar to anyone that followed the AOP landscape, even from the distance; not a bad talk for anyone that is just starting out. one of the best AOP talks last year was on step-by-step adoption techniques with cookbook examples. i think this is the best way of introduction, since a lot of people are just paralyzed by the power available to them, not knowing where to start.

next i stopped by venkat's "refactoring your code" talk. i remember him being an excellent speaker, and i primarily wanted to see how he explains this already familiar topic - perhaps copy a trick or two to use myself if i were to speak in front of others. instead i was simply drawn in and followed every moment of the presentation. he was coding at breakneck pace, and yet amazing lucid, humorous, and illustrative, tying it altogether with the agile development principles. the examples were excellent; he was getting feedback from the audience, and yet driving the talk; setting up little traps and problems and solving them effectively, illustrating the point. his real-life anecdotes are so vivid often crossing the boundary into absurd; and they stick in your mind, together with his deadpan delivery. he is so natural, none of this stuff is forced or awkward. excellent talk; as a teacher he is unsurpased. a nice example of repetition as a stepping stone that gradually lets one refactor methodically and safely. some quotes from william zinsser on writing. and some famouse quotes (perhaps if i write them down, i will actually remember them):

perfection is achieved, not when there is nothing left to add, but when there is nothing left to remove. (Antoine de Saint-Exupery)
Programs must be written for people to read, and only incidentally for machines to execute. (Abelson and Sussman)
unit tests and code is like my left leg and my right leg - i do not walk three miles just on my right leg and then switch

this set the theme for venkat's talks for the rest of the weekend - i attended his talks on seemingly familiar topics, but he managed to either illuminate them from a different perspective, or really arrange them properly in my head, clarify and organize them. i seem to get a lot of stuff on very intuitive level, and then it takes me a while until i can clearly formulate it. teaching would probably be the best way to get it organized, but venkat's talks is a nice shortcut for that. some of this stuff is also available on his website.

still under the impression from the books i've read, i went to see bruce tate's talk on effective teams ("effective teams: the dirty little secret"). it was pretty decent, although a little tedious at times (when we had to remember what the letters were corresponding to the personalities and then go through the examples). up to this date the best books on team organization were "peopleware", "rapid development", two joel books ("best in software writing 1" and "joel on software") and many-many blogs on interviewing (stevey's drunken blog rants is one of the recent favorites). so a lot of the stuff was already familiar, although it was nice to see a distillation of personalities (rock vs. visionary, as well as myers-briggs personality test), as well as reaffirmation that good teams need both (here it is also pertinent to mention the difference between a developer and a programmer, and where each one fits). another good point - mentoring (get one; perhaps even outside of company) and keeping up.

looking through the slides for the first day, i realized that glenn vanderburg had a very nice talk on collections api - filled with practical details, very hands on, very useful. i think this goes for all of his other talks that weekend - very well researched, very thorough, very technical, in retrospect, i wish i have attended more of them.

quotes 1

Posted by anton
on Tuesday, May 16, 2006
i really should be using a wiki for this, but oh well. this just helps me to remember things as i read. and some of it, sadly, is a reminder of what i used to know, but what fell out of the context and became forgotten.
Conway's famous law: "Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations."