Thin Slice vs. Layer-wise, Or Depth-first vs. Width first

Agilists always advocate for incrementally developing a thin slice of the application, in the form of a user story, that implements end-to-end changes that are required to deliver business value to the customer. This will normally entail changes to the UI, front-end logic, back-end components, databases, service interfaces, etc. This process results in the end user receiving value for every story delivered. The customer gets this value early, while developers have to manage only one small change at a time, and significantly reducing code integration overhead.
But what about efficiency?
Some will argue that working on different layers requires different skills, and that there is a waste incurred in task switching, e.g., switching from working on the UI to working on the back end. It can be demonstrated that working width-first, as in developing 10 UI components (A1, A2, A3, ...), , then 10 back end components (B1, B2, B3, ...), etc., will require less time than working depth-first (A1, B1, ...), then (A2, B2, ...).
But then, who's right?
In fact, both are. It is indeed more efficient to work on the same task type, since there is less task switching, and there are mass production techniques that could be employed. However, the customer gets value much later, process feedback is delayed, and there are more integration work and problems that are only discovered late in the process.
It so happens that in software development, the Agile method is far more superior. The benefits of an Agile process increase in line with the rate of change in the process.
But what about efficiency?
To the non-agilists, it should be noted that efficiency isn't the point. Thinking mainly about efficiency distracts from the goal of delivering more value to the customer, at a faster pace. To concentrate on efficiently producing UI components, for example, increases waste by delivering more than the customer needs, by increasing the rework required to adapt to changing/better understood customer needs, by delaying feedback, and by increasing work in progress. This is a form of local optimization that negatively impacts the overall goal.
But also, to the agilists, the efficiency argument should be understood. There will be opportunities during development to queue up similar tasks, and leverage specialized skills, and mass production techniques, to gain immediate task efficiency.
The idea, then, is to never let efficiency distract you from your gaol, and to only pursue it if it doesn't increase work-in-process, and doesn't delay feedback or customer value.

WebSphere Trace, Spring, commons ToStringBuilder, Hibernate, and LazyInitializationException

We've seen the exception below, because WebSphere diagnostics and tracing was enabled.
We disabled it by editing the logging properties from WebSphere console/Logs and Trace/select your server's/select Diagnostics Trace Server/Runtime/click Change Log Detail Levels, then editing logging information, and removing unneeded packages after "*=info:" ...

Stack trace:

[2009-07-23 15:56:10,078] WebContainer : 1651 org.hibernate.LazyInitializationException ERROR - could not initialize proxy - no Session
org.hibernate.LazyInitializationException: could not initialize proxy - no Session
at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:86)
at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:140)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:190)
at com.mycompany.myproject.domain.MyObject_$$_javassist_20.hashCode(MyObject_$$_javassist_20.java)
at org.apache.commons.lang.builder.HashCodeBuilder.append(HashCodeBuilder.java:452)
at org.apache.commons.lang.builder.HashCodeBuilder.reflectionAppend(HashCodeBuilder.java:413)
at org.apache.commons.lang.builder.HashCodeBuilder.reflectionHashCode(HashCodeBuilder.java:379)
at org.apache.commons.lang.builder.HashCodeBuilder.reflectionHashCode(HashCodeBuilder.java:155)
at com.mycompany.myproject.domain.DomainObject.hashCode(DomainObject.java:14)
at java.util.HashMap.hash(HashMap.java:324)
at java.util.HashMap.containsKey(HashMap.java:470)
at java.util.HashSet.contains(HashSet.java:207)
at org.apache.commons.lang.builder.ReflectionToStringBuilder.isRegistered(ReflectionToStringBuilder.java:135)
at org.apache.commons.lang.builder.ReflectionToStringBuilder.appendFieldsIn(ReflectionToStringBuilder.java:660)
at org.apache.commons.lang.builder.ReflectionToStringBuilder.toString(ReflectionToStringBuilder.java:867)
at org.apache.commons.lang.builder.ReflectionToStringBuilder.toString(ReflectionToStringBuilder.java:339)
at org.apache.commons.lang.builder.ReflectionToStringBuilder.toString(ReflectionToStringBuilder.java:173)
at org.apache.commons.lang.builder.ToStringBuilder.reflectionToString(ToStringBuilder.java:124)
at com.mycompany.myproject.domain.DomainObject.toString(DomainObject.java:24)
at java.lang.String.valueOf(String.java:1505)
at java.util.AbstractCollection.toString(AbstractCollection.java:469)
at com.ibm.ws.webcontainer.srt.SRTServletRequest.setAttribute(SRTServletRequest.java:488)
at org.springframework.web.servlet.view.AbstractView.exposeModelAsRequestAttributes(AbstractView.java:337)
at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:206)
at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:257)
at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1183)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:902)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1095)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1036)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:118)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:832)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:679)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:565)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java:321)
at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:236)
at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:257)
at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1183)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:902)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1095)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1036)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:145)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:119)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:55)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:186)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:832)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:679)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:565)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:748)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1461)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:118)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:458)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:387)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:102)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1473)

Analysis

It seems that enabling logging for the package com.ibm.ws.webcontainer causes WebSphere to log request attributes. During view rendering, the call to request.setAttribute(), in Spring's AbstractView.exposeModelAsRequestAttributes() , which is implemented by com.ibm.ws.webcontainer.srt.SRTServletRequest.setAttribute(), attempts to log model attributes by calling toString(). We add our domain objects as values for model attributes, and our DomainObject implements toString() using Apache commons ToStringBuilder, which recursively walks the whole object graph, causing all references to be initialized. This all happens outside the boundaries of a transaction, and hence we get the exception above for our lazily loaded references.

Now this should be searchable, just in case ...

The Mythical Man-Month

(So at last I read it ...)

By Frederick P. Brooks, JR.
An enjoyable read indeed. The book is for those with passion for software development, from someone who shares this passion.
The book contains a good deal of timeless advice, although one might wonder how much of the book is relevant today. I'd offer that most of the book is. There has been times when I was puzzled by the content, and totally missed references to the machines, tools, and procedures. Nevertheless, it's amazing to see how much had changed, yet how much really didn't. In that regard, the discussion around essential (irreducible core complexity) and accidental difficulties (those pertaining to technology limitations, etc) is especially illuminating.
Here are some useful pointers:
The authors homepage
On Amazon

Book: Radical Project Management

(Note: this is a personal memory entry)
(Note on Note: More on this note later)

The book "Radical Project Management" by Rob Thomsett, is very interesting. It introduces a lot of good practices, that are indeed in wide adoption today. I'd recommend it, if not for the actual practices, then for the underlying message to project managers.
The Thomsett group website: http://www.thomsett.com.au/.
Supporting material and articles can be found here: http://www.thomsettinternational.com/. Of note are Project Pathology, and Estimation Games. This is not to discount other articles or the site, but these were specifically mentioned in the book.

The Ultimate Continuous Integration Tool

Warning: this entry is out there. I can not be blamed for wasting your brain cycles. However, by taking things to the extreme, some value may be gained.

I'd like to continuously integrate all the code I write, and I want this to happen very continuously. And that would happen if my code is integrated on every key stroke.
My IDE, the build machine (CI server), and the source control system will be very smart. Every time I type key on the keyboard, they will mark a revision of the full code base and start running the build. Let's call this code revision a candidate revision (CR). The build, of course, includes compiling the code, running all unit tests, perhaps even functional tests, and whatever else we want to include. If the build passes, the CR is automatically checked into source control. If the build doesn't pass, then the whole thread is ignored. Obviously, since I won't stop pounding on the keyboard, the very powerful build machine will be running multiple builds at the same time. After all, the computers are not yet that advanced, that we should expect the build machine to be able to run builds instantly. Oh well.
At the same time, the system will be doing the reverse: It will be continuously merging all successful revisions from source control (SC) into my local code base, since everyone else is so productively writing code at the same time as I am.
How could this work? Well, my IDE will be so well integrated with the super powerful CI server. The CI server will be the single authority deciding whether a build is passing, and thus would be checked into source control. The CI server will be running many builds at the same time, but will act as if all the changes occurred sequentially.
The CI server will be receiving a stream of candidate revisions (ok, just deltas) from multiple developer machines. It always starts from a successful source code revision (CR0). Based on the order a candidate revision (CR1) is received, it will run the build and determine if it should be checked into SC. At the same time, the CI server is also processing other candidate revisions (CR2, 3, ...) from myself and other developers. The CI server will be running multiple builds at the same time: a duild for CR1, another for CR2 (on top of CR1), another for CR2 on top of CR0, in case CR1 fails, and so on.

Here is a diagram of the process:

This would be really nice. Don't you think?

ssh connection with public/private key pair not working on Leopard?

Solution: try passing the private key file to the ssh command using -i:

ssh -i identity_file user@server

The problem
I tried setting up a DSA public/private key pair on Leopard. but I didn't accept the default private key file name. Leopard didn't prompt me for the passphrase, and instead was prompted for the normal username/password.

Project Development Knowledge: Sharing and Enduring

The previous entry introduced the problem. The following is a discussion around practices that can help address it.

Standups

Daily standup meetings [1], aka daily scrums, are a great way for the team to share information about the current tasks being implemented by different team members. It's a quick forum, at the end of which, each team member will have basic knowledge of what other team members are up to. This can be very effective in eliminating duplicate efforts, as well as helping team members relate to other work affecting their own. Team members who notice the potential for cooperation and knowledge sharing during the standup, should get together after the meeting to carry on with further, more detailed examination of their work.

Pair programming

Controversial as it may be, this practice is very effective in achieving higher team productivity through continuous knowledge sharing. The risk of the knowledge leaving the team is greatly reduced. And if we enhance this practice by frequent pair rotation, it's even more effective, as it spreads more knowledge throughout the development team, and it encourages cross-pollination of ideas.

Colocation

Having to communicate information to other team members highlights a hand-off transition, that is better avoided. It's much better if all the team is there, witnessing and participating firsthand the effort underway, rather than being communicated what happened. Colocation of the team, or as much of the team as possible, is very effective at eliminating that need. And when information is communicated within a co-located team, it's at least an order of magnitude more effective, efficient, and complete, compared to other means.

Code reviews

When a pair spends a day modeling a piece of functionality, or refactoring a key part of the system, they are likely to want to tell other team members about it. Invariably, everyone in the team is interested to know what others are doing, and how certain problems are being addressed. While pair rotation helps here, getting the whole team to participate in a code review session can achieve some of this benefit to a wider audience. It's also a great forum for seeking guidance, sharing opinions, and exploring novel ways to address issues.

Automation

Automating a certain task is an excellent way of sharing how it's done. Automation enables other team members to achieve the task, as well as serve as a documentation on how to accomplish it. For example, instead of me asking you how you query for the balance sheet, I can either use an automated script to get the information, or I could learn from the script how it can be done.

Self documenting, readable code

Code can be considered a misnomer in this regard. We'd like to have code that does not require deciphering. We'd like to be able to know what the code is doing, and how it's done, clearly, with minimal effort. Herein lies an argument against clever programming techniques, that make it harder to reveal intent and side effects. Use code reviews to highlight less than obvious techniques, and have the team workout what it's most comfortable with. For example, if multiple team members are having issues with a piece of code that uses reflection, consider first informing more members how the piece works, and if necessary, consider an alternative implementation. Enabling higher team effectiveness is valued higher than programming cleverness.

Good check-in comments

These can go a long way in telling the story of project development. A check-in delta tells you what changed. The comment tells you why. When a developer has to go through source code history to understand the rationale behind a certain change or design decision, these comments can be very helpful in that regard. Try to be helpful to the consumer of the comment. For example, instead of typing "implemented story #123", provide more information: "Story #123: added capability to classes x and y to access context z to achieve a and b."

Wikis

Project wikis have been around for while. Dare I say that they are the norm these days? Wikis are an excellent source of day-to-day knowledge needed by the team. For example, database connection information, URLs to local servers and useful documentation, pointers to tools, project and domain specific acronyms, etc. They are also useful in documenting repetitive development tasks, that are yet to be, or are difficult to automate.
One particular use that I wish to highlight is documenting errors that are encountered during development, whether or not they are corrected. For example, if while starting a server we notice that we are getting an exception, we should start by documenting this fact. We then add the solution once we figure it out. Certain problems have the tendency to re-occur, while some are unlikely to occur once fixed. Nonetheless, a similar issue might arise and the fix may prove useful beyond the original use.

A searchable, archived email list

I have not seen this one on many projects. However, a sizable portion of the project information is exchanged using emails. For example, feature discussions between developers and BAs, the resolution of design issues, technical announcements, in the form of the introduction of a new build tools or code libraries, and project course-changing decisions, to name a few. An automatically archived project specific email list that is searchable can be very valuable for future development. Using this knowledge base can aid in understanding why things came to be the way they are.


This list isn't exhaustive, and each one deserves its own discussion. I meant to hint and introduce. None of these practices preclude another, and you can attempt all of them.
A bit of warning though: do not over do any of these, and keep the practices light. After all, we are after agility. Don't let these, or any other practices slow down the gemba.

As usual, I welcome, and appreciate, the reader's feedback.


[1] http://martinfowler.com/articles/itsNotJustStandingUp.html