Emacs org-mode goodness

I had used Emacs Org mode in the past but not the extent where it is now tightly integrated with most of my workflows. Yes, I’m an Emacs fanatic.

At home, I do lots of things in Emacs (programming, News feed reading, IRC chat, Blog posts, time tracking, tasks tracking and planning, Revision control tasks, Hudson/Jenkins monitoring, general purpose editing, etc.).

Org mode is getting more and more integrated into my workflows.

While I haven’t mastered Org mode yet, I use it for the following purposes:

  • Capture notes or sprint tasks
  • Writing emails
  • Writing blog posts
  • Managing Links/bookmarks with org-protocol and org-capture
  • Documentation work related investigations with a combination of other tools for graphs, etc.
  • General planner with Org agenda to track tasks

Notes and Work related tasks

Sprint, scrum and other notes

At the beginning of each sprint, I do the following:

  • Create a master Sprint file to keep track of all the tasks with associated TODO items and time spent.
  • Create a Sprint specific directory with additional folders grouped by topics

I try to use time tracking (clock-in, clock-out) as much as possible for any task that I’m looking at, using a set of Org-capture templates.

Some of my setup for Sprints and notes were inspired by the following articles:

Investigations or other analysis

Again cudos to Howard for this great blog posts and Youtube videos!

Using the links below as inspiration, most of my work is now well-documented enough and in plain-text!

The great thing now is that I’m able to both document and capture results of my investigation at one place. For a given investigation, I might be gathering details from a set of different sources (Web Services, Databases, other tools).


For my own corporation (Rimero Solutions Inc.) email account, I rely on Gnus, an Emacs mail client.

Please visit the official Gnus Homepage , if you want more details.


Tool Purpose
Gnus Mail browsing
Isync or OfflineIMAP Mail synching
Mairix Mail search
BBDB or Org Contacts Contacts management
msmtp Sending the actual emails outside Gnus
EUDC Contacts lookup from LDAP if needed


I keep a set of email snippets in my ~/.emacs.d/snippets directory.

total used in directory 24 available 62731738
drwxr-xr-x  5 yves  staff  170 Feb 10 18:00 .
drwxr-xr-x  3 yves  staff  102 Jan  4 03:56 ..
-rw-r--r--  1 yves  staff  312 Feb 10 18:00 autoinsert-mailgeneric
-rw-r--r--  1 yves  staff  376 Feb 10 18:00 autoinsert-mailinvoice
-rw-r--r--  1 yves  staff  781 Jan  4 03:57 tplsprint

Managing credentials

I keep encrypted Org mode password files for few services that require authentication. In the past, all that information including the tiny details (IP addresses, URLs, credentials) were in my head. Sometimes, but not often, I’d forget after a while…

On my OS filesystem, there are couple of encrypted Org files per project or area of interest, using PGP and Org Crypt.

Each time that I save a passwords file, all the sections that are marked as private get automatically hidden and encrypted via PGP.

Managing Links and bookmarks

At home, while I try organizing carefully my bookmarks with nested sections, on contracts I need to keep track of a set of project driven links or references.

I use a combination of tags with org-protocol and org-capture to store Links from Google Chrome.

I do not have as much time as I wish to read news or articles about technology, software architecture, software design, coding, project management, etc.

When I see something that is worth revisiting:

  • I store a link to the Web Page in my Links file
  • If the link is associated to a pending task, I add a tag and a TODO item for review later

Blog posts

This blog post is written within Emacs itself, using the org2blog package.

Both my corporate and personal blogs are powered by WordPress.

The org2blog package works pretty well with WordPress.

Once I’m satisfied with the blog post, I can post a draft to WordPress and later on publish the post after a quick review.

Documenting investigations and other work

When I’m doing  personal or corporate research, I try documenting what I’m doing and collecting enough information, in case I need to share details with other people or recall what I was previously working on.

Tasks planning

I just navigate through Org agenda to see what was acomplished during the last 2 weeks and what’s still pending.

Changes on a Production System

I was thinking about the various updates that an Application that is already deployed in production require:

  • Bug fixes
  • New features
  • Configuration changes
  • Re-design because of bad/incomplete requirements or code.
  • etc.

Is your application/program without bugs?

  • If you tell me that the application is in Production with zero bugs, I have a problem with it.
  • If you re-phrase it by saying that there are no known bugs, I still might have a problem with. Maybe the system isn’t tested enough?
  • If you say that given a test cycle of XXX weeks, there are 10 known non-critical bugs that will need to be addressed, I think that’s a more reasonable answer.

If your application really has no known bugs, with reasonable testing, it’s often an indicator of a new application or a program that no one is really using… The application might also be a plain hello world.

Below is a small subset of usual issues when working on Production code/configuration changes.

So, there’s a problem for YOU to fix

Sure, is there really a problem?

When things don’t work the way that we expect them to, we say that there’s a problem.

As we become more experienced, we say that there might be a problem, and when there’s no clear understanding, we say that we’re investigating. Even with many years on the job, we still might be wrong, but we’re likely more careful.

Often, a reproducible scenario can be presented, sometimes not.

Bug reporter: It doesn’t work. I didn’t do anything special.

Me: I looked at the logs, they’re empty. As the problem cannot be reproduced and there are no details, it will automatically be closed within the next 2 days. Thanks for wasting my time btw…

Given the above example:

  • Occasionally the user will try to provide enough information when reporting an issue. Should the user get more education about bug reporting?
    • Environment, application versions, etc.
    • Scenario with steps to reproduce.
    • What exactly is not working? Expectations v.s actual behavior.
    • Is it blocking or is there a workaround at this point?
    • etc.
  • An empty log and a non-reproducible problem doesn’t mean that there’s no issue… The issue itself might manifest itself at specific times or under specific conditions that are difficult to find out.

Many times, there’s no problem at all:

  • The user’s knowledge of the system can drive him to wrong expectations.
    • How can we help the user? Is our system coherent?
    • Do they require more documentation or UX changes?
    • What else can we do?
  • An external factor maybe come into play impacting your application: You learned after the fact that the system will be down for 5 hours…

You start looking immediately at the code

Before looking at the code:

  • Can the problem be reproduced?
  • Is it maybe related to bad data/configuration?
  • Is it maybe related to external systems/processed?
  • Is it maybe something else that is not code?
  • Is it maybe related to code?

Lots of time can be wasted, by jumping directly to the code and debugging issues based on wrong data or wrong configurations.

It’s easier to try eliminating non-code related concerns first. Even if you missed something, the code might help you understand it later.

Accidental operations

UNIX/Linux systems – Files deletion

You wake up one morning and you learn that a bunch of files that have been deleted. What happened?

The usual conversation with stakeholders

Stakeholder: What happened?

Me: I didn’t do anything, I was on support though…

Stakeholder: What happened then?

Me: I don’t know man, go figure… I may not even care…

The reality

Someone or a program did delete files. Depending on the system configuration and logs it may be possible to trace all the operations.

Who will come forward or analyze the information for explanations?

What do we do?

Do we have a backup? Sometimes we just don’t for many reasons…

Databases – SQL client mistakes

You’re connecting to a database. The network connection is too slow and you clicked too quickly.

Few seconds later, you realize that a popup showed up and disappeared immediately. The Delete database menu item got pressed without confirmation, something just happened.

For whatever reasons, you had full admin privileges, so what might have happened did really happen….

What do we do? Be prepared to answer the question Do you have a backup?

Even if you did have a backup, what happened between the database updates and now?

  • Can your backup restore the database to its previous state, in addition to all the new changes?
  • How do you perform database restore/synchronization? How are those DB servers configured (clusters/RACs, etc.)

Code verifications

How do you validate new code changes before pushing it to Production? Even if you’re cautious, you’re bound to make errors once in a while, but you should always take as much time as you need/can.

Me: I merged the code and it’s all good. It took me 5 minutes and it’s checked-in!

Stakeholder: OK, great job! Let’s push it to PROD within the next 5 minutes.

Code merges and code changes for an application which is in Production, shouldn’t be a matter of 5m. Why? Because a basic mistake, misunderstanding or regression bug can create bigger problems, than just doing no changes.

What can happen here?

  • You make a very minor code change (one liner)
  • Someone has updated the functions or libraries driving your code changes (with or without your knowledge)
  • You think that you’re done after 5 minutes, but you’re not… You may not have time to test, but you should still test it anyway.

Just re-test, especially when you don’t have full control over what’s happening.


If you have the exact information, whether or not it’s about configuration or code changes, it’s better to use copy/paste whenever possible.

Many people, try to type manually information that they see on the screen. A simple typo, even an additional space can have drastic consequences…

If you’re a copy/paste person, do not copy examples that do not make sense for the current task. Update placeholders or variables using common-sense. When common-sense still doesn’t make sense, ask questions.

Configuration changes

Wrong instructions

Someone sent the wrong instructions for configuration changes. This can happen to anybody.

It’s simple enough to review notes and/or ask for someone else’s to review.

The next questions are:

  • What’s the plan?
  • What’s the time frame?
  • How can we verify those changes?
  • When will those changes be pushed?

Taking full-ownership of Production systems is a NO NO

The client want you to take full ownership of systems and their configuration, you really shouldn’t. By system, I mean a set of integrated components that is not just about your particular software application, in the general sense.

Those systems are usually not related to only your specific project applications and servers. There’s other stuff out there that you don’t know about, that can impact directly/indirectly your project.

After many issues on a both non-production and production systems, I was often given the opportunity to get full access in order to setup systems from scratch (software, servers, etc.).

We’ll give you a blank UNIX operating system, with permanent root access, for you to setup.

You’ll configure everything from the network, firewalls, LDAP servers, Databases, applications, etc.

That way, if something is not correct you’ll know it and fix it. By the way, you’ll take the hit for any issue…

In big companies systems are way too complex.

You’ll never have a full/deep/clear understanding of every system and application at a given point in time.

Just focus on what is within your control, your specific application/project and let other specialists deal with external systems that are beyond your knowledge, interest, focus, area of expertise.

You should request help for external systems if you need some support.

Clojure CLI adventures

Recently, I played around with Clojure in terms of porting a set of scripts to a cross platform command line tool.

This was fun, as I had no experience writing actual Clojure applications, just small exercises. I also don’t have a full understanding of the language (complex macros, core.async, other topics that I’ve yet to explore).

The end result of my Clojure experiment was good, reasonably fast and works reasonably well (Not stressed-tested much though). I’m looking forward playing some more with Clojure whenever possible (more POCs or production applications). I encountered couple of oddities but I was able to get through most of them without intense brain activity.

Command line options and arguments parsing

tools.cli is very straightforward and very idiomatic from a Clojure perspective. Its seems fragile, maybe not that popular? I don’t know.

Plumbing custom validation functions didn’t work for me, did I get it wrong?? -> It looks like the support for multiple validation functions is not yet in an official release. http://dev.clojure.org/jira/browse/TCLI-9

Modelling configuration

There are plenty of ways to deal with user configurations:

  • Java properties
  • Some YAML
  • JSON
  • Groovy code using ConfigSlurper for Groovy applications
  • Custom DSL
  • Environment variables
  • etc.

For Clojure I went straight with EDN, code as data, plain hash-maps with few lists and dynamic functions which are evaluated at run-time for object model specific features.

Handling exceptions

In java, I often prefer to create business exceptions. I also like segregating internal errors from external issues (third-party application crash, etc.). This requires couple of classes, sometimes error codes, details about what is happening and if possible how to fix it, etc.

slingshot is a nice library for throwing/catching exceptions in Clojure. No need to create specific dedicated classes that don’t contain much…

(throw+ {:type :external-error :reason "Failed to..." :next-steps "whatever needs to be done" :useless-stacktrace-for-the-user "NullPointerException"})

Handling external processes

I first started with clojure.java.shell, but I noticed quickly that some processes were hanging, plus other annoyances. I then looked at conch, similar problems. Even basic operations seemed to hang once in a while, in addition to known CPU/Memory intensive tasks. CLJ-959 is one nasty bug that I encountered, soooo nasty… Processes seemed to have completed execution and yet my application was was not exiting…

I was left with no other choice than falling back to the Java ProcessBuilder class.

You start an application and print its output as it comes, (i.e. rails server or whatever) then you hit Control-C. What just happened? That server is still running and the process associated to it was not killed?? Instead of using reflection and accessing the process information (UNIXProcess, Win32Process), I used the set-break-handler! function which registers a signal handler for SIGINT. In pure java that would be sun.misc.SignalHandler logic. JVM shutdown hooks just didn’t cut it for destroying sub-processes prior quitting the program.

Handling IO (symbolic links, folder deletion)

While it’s possible to invoke command line tools such as ln or mklink (Windows), a more interesting alternative is to interact with the OS via JNA (Kernel32 for Windows and C library for Unixes).

Also under Windows, all you need from the user is opening a shell with Admin privileges (custom VBS script to open cmd.exe with admin rights), if your application creates symbolic links or requires some specific privileges.

Clojure JNA really makes life simple.

I/O operations using pure Java tend to be slow especially under Windows (Try deleting a huge folder with pure Java, whatever smart graph traversal approach).

Avoiding state

It’s very tempting to introduce static/global variables. You need it, there you have it, you can easily call it from anywhere in the code. Those who have suffered from Singletons know the dangers of introducing some potential architecture/performance limitations.

Not abusing classes/records

As a Java developer, I do have to handle other programming languages once in a while (Groovy, Python, etc.). Perl, humm… no thanks. I tried using Clojure records only when I thought that I needed some kind of grouping.


I just went with shell scripts to wrap some fat/uberjar vs. creating a JSmooth or Launch4j executable for Windows and playing with makeself for Unix/Linux. Those wrappers just delegate to jar -jar somejar and pass all your command line arguments to the main class.

Recalling a project with VRaptor.

I came across an article on Oracle’ Website today about JSR 371. I was surprised to see VRaptor Web Framework mentioned. It seems that VRaptor is known by a reasonable amount of people, that wasn’t the case about 10 years ago.

I got to use VRaptor only once and it was in 2004. I was doing a part time contract as Java teaching assistant for BAA and MBA courses. My tasks involved a small Web application in addition to helping the professor and students. As soon as the semester started, we needed a solution to host the student’s labs(Java Web applications) on the server, in addition to rebuilding the server itself.

Project requirements and environment setup


  • Setup a Linux server with SSH access using Debian GNU/Linux.
  • Host the students Web applications that were to be deployed for labs.
  • No student would be having access to the Tomcat console to avoid mess and unrecoverable mistakes.


  • Headless Debian GNU/Linux on an IBM box with either 128MB of RAM or 256MB. I don’t recall the exact available memory.
  • An FTP server (I used ProFTPD).
  • SVN for version control.
  • MySQL as database
  • CRON jobs for backups
  • Clamav for anti-virus scanning.
  • Tomcat as Java application server.
  • Shorewall as firewall.

Web applications control panel for users

The next challenge was building some kind of control panel for each student with the following features. That was needed to help student deploy their labs without having access to the Tomcat administration console.

Below were the most basic features that needed to be available:

  • Login
  • Upload and deploy a war
  • Start, stop an application


The main project tasks consisted in the following:

  • Setting up the Linux box.
  • Automating the students accounts(FTP, MYSQL, SVN, Tomcat, etc).
  • Creating a Web application to help students manage their labs (Usually Struts or similar Web applications).

I’m going to skip the part related to the Linux setup as there was nothing more than following wizards, being aware of some tools and editing some configuration files.

For about 200 small Websites and a maximum of 256MB of RAM, we run out of memory once in a while and often near the end. At some point, Tomcat was restarted on a daily basis via a CRON job.

Writing the code

I had about 2 weeks to setup the Linux box and basic tooling, in addition to writing the Web application iirc. I definitely couldn’t afford too fancy or too complicated tools/technologies in order to get things done.

Setting up students accounts

After evaluating some Shell scripting, Python and similar languages, I settled with Apache Commons Jelly.

Jelly was the core scripting language for Apache Maven until Maven 2.x. I picked Jelly for the following reasons:

  • Easily embeddable and written in Java.
  • Declarative using XML and externalizable. Pass an XML Jelly script as argument to a custom Java program and you’re done.
  • Many available sub-libraries (SQL, SOAP calls, Swing, etc.)
  • Some Apache Ant style DSL for tasks.

A set of Jelly scripts were created to perform accounts setup tasks:

  • Process the list of students from an MS Excel spreadsheet.
  • Generate the FTP accounts in the PROFTPD database.
  • Generate some MySQL users and related tables.
  • Generate some Linux and SVN accounts.
  • Generate some virtual hosts for Tomcat (XSLT processing of Tomcat configuration). Depending on the course, each student had a maximum number of Web application contexts.
  • Send emails with the account details and Web application control panel instructions to each student.

Writing the Web application

After setting up Tomcat with jsvc to run on port 80, I had a bigger issue. What will I use as Web application framework?

  • Struts (I didn’t and still don’t consider it a viable option). It has a nicer website though.
  • Pure servlets? The application was small but using only servlets could have been painful.
  • Something else? I wasn’t that familiar with the Java ecosystem in terms of Web Frameworks. I think that Tapestry existed already, but it seemed just too complicated for no good reason.

I had some “OK” memories of ASP and JSP felt pretty much similar even if I think it’s cleaner with custom tags. My only sweet Web experience with Web applications was with ASP.NET (C# and VB.net). I was very pleased with the design, architecture and documentation of ASP.NET.

While I’d likely never had used VRaptor for a medium size to big application, it was an attractive choice for something simple. I was googlin’ for some Struts alternative and I found VRaptor’s Website by accident. After 5 minutes of reading, my decision was made. I knew that building the Web application was going to be painless and without too much configuration.

The Web application used the latest version of VRaptor at the time, Hibernate and few other technologies, I think that Spring Core was one of them mainly for Hibernate integration. I picked Apache Velocity instead of JSP for the view layer.

Students authentication was performed by a controller/action against a MySQL database, no JAAS or alternative. Related Web application management tasks were executed using the Apache Ant API and Tomcat Ant tasks. Some basic AJAX was in place to avoid multiple simultaneous clicks that could put too much pressure on the server.

Writing this Web application was pleasant. There wasn’t that much functionality or fancy UI controls but the development didn’t take time. The application pretty much “worked”, did very few things and did them well.

Functional programming and the JVM

Couple of years back, I couldn’t imagine writing a real world application using a JVM language other than Java. “Please use other JVM languages just for plugins or extensions…”. Dynamic JVM languages seemed very good at writing plugins and also very slow in terms of general performance.

I recall the early 2000’s when I was learning .Net(mostly ASP.net with C# and VB.net). So many languages were available, while not officially supported. Nowadays F# seems to actually be used. Java got JSR 223 a while ago, and some JVM languages (other than BeanShell, Jython, JRuby) ‘appeared’ from everywhere.

JVM languages previous nightmares

My worst experience with other JVM languages happened while implementing some scripting support (Swing oriented) with XML as main markup.

Between dynamic proxies, binding/unbinding variables, code plumbing to hook dynamic code, … it was hell. I figured that it was easier to just write Java Swing code instead of thinking about having something similar to XUL.

I spent couple of weeks implementing a ‘framework’ to a point that it became actually usable (Jython or Rhino for scripting). The application was slower with too much ceremony for non trivial business logic and decoupling issues.

Java it is, a class for a function

I’ve been looking at functional programming time once in a while, reading articles here and there… Lately I decided to give it a decent try. Many times in Java, we’re compelled to write classes just because we need functions more or less logically grouped.

Compared to many other languages, we cannot pass functions as we please. While that makes the code simpler, some boilerplate interfaces are generally in the mix, just for callbacks while intending to keep it ‘generic’.

State and singletons

Many concurrency issues are often related to sharing too much state and not anticipating that an API client might decide to use some code inside threads. When it’s cheap to create and not immutable, it’s probably better not to share. Copy constructors or something are useful when passing objects arounds. The general problem is when people implement tons of singletons (flexibility concerns, possible synchronization issues, testability, etc.).

Selecting clojure over others

I selected Clojure over Scala, Haskell / Jaskell and other non JVM hosted functional languages.

Learn You a Haskell for Great Good! seems to be a good book for Haskell but it would take me lots of practice to program anything remotely useful, including writing idiomatic Haskell code. I read few pages and tried few exercices, but eventually I gave up…

Scala is also nice and probably easier for me to learn, but I don’t really like the syntax… It reminds me a bit of Flex/Actionscript, mixed with couple of other things from other programming languages. It looks like it’s easy to create a very cryptic API and preserve job security.

Clojure is a very interesting language to me for many reasons:

  • I’ve picked up more interest in LISP dialects after using Emacs heavily once again.
  • The syntax is not too complex (Some special constructs/forms, mainly about lists and learning core functions, macros).
  • Data structures are immutables by default, unless you need state or if you’re heavily using Java interop.
  • The development is driven by smart people and the community seems dynamic.
  • It’s fun to write Clojure code and to have a very tight REPL/’interpreter’ interaction.
  • IDE and text editor support (VIM, Emacs, etc.) is decent.

I’m still learning by actually writing code, while trying to remember concepts on the fly. I find it easier to just get my hands dirty instead of reading a whole book before doing anything.

When reading code examples or watching screencasts, I apply few variations and learn new tricks. I have few project ideas in mind once I get really comfortable with Clojure.

Review of the Grails Gradle Plugin

UPDATE: Using the bootstrap configuration for the Tomcat plugin excludes it from the grails-war task.

See https://github.com/grails/grails-gradle-plugin/pull/43

Gradle support for Grails is maturing slowly and I must say that I can’t wait for Grails 3.0. “Oh yeah, I’m excited!”🙂, almost.

There are still few bits that I’m not clear about though in terms of how tight the integration will be, not specifically from an IDE usage perspective.

I watched a presentation from Luke Daley (aka alkemist) on Youtube(gr8conf 2013). It showcased the Gradle plugin for building Grails applications using the grails-gradle-plugin.

I was able to create a small POC and I want to share that experience with you.

General notes

Building a war and running Grails commands? Not a problem.
gradle grails-run-app
gradle grails-war

You can configure the Grails environment using -PgrailsEnv as command line argument.
-Dgrails.env= or -Pgrails.env= doesn’t seem to work.

Arguments can be specified using -PgrailsArgs
gradle -PgrailsArgs=’com.Domain’ grails-create-domain-class

For some reason, the grails-gradle-plugin seems to require a closure with a Grails version specified twice (assuming that version is only used for bootstrapping the initial call??, while grailsVersion is used for building). I think that it should probably be consolidated…

grails {
grailsVersion '2.2.3'
version '2.2.3'

Below is a build.gradle file that does work for the Grails Gradle plugin 2.0.0-SNAPSHOT. Dump the file into some folder and run gradle init first.

buildscript {
repositories {
maven { url 'http://repository.jboss.org/maven2/' }
maven { url 'http://repo.grails.org/grails/repo' }
maven { url 'http://repo.grails.org/grails/plugins' }
maven { url 'http://repository.springsource.com/maven/bundles/release' }
maven { url 'http://repository.springsource.com/maven/bundles/external' }
maven { url 'http://repository.springsource.com/maven/libraries/release' }
maven { url 'http://repository.springsource.com/maven/libraries/external' }

dependencies {
classpath 'org.grails:grails-gradle-plugin:2.0.0-SNAPSHOT',


apply plugin: 'grails'

repositories {
maven { url 'http://repository.jboss.org/maven2/' }
maven { url 'http://repo.grails.org/grails/repo' }
maven { url 'http://repo.grails.org/grails/plugins' }
maven { url 'http://repository.springsource.com/maven/bundles/release' }
maven { url 'http://repository.springsource.com/maven/bundles/external' }
maven { url 'http://repository.springsource.com/maven/libraries/release' }
maven { url 'http://repository.springsource.com/maven/libraries/external' }

grails {
grailsVersion '2.2.3'
version '2.2.3'

configurations {
all {
exclude module: 'commons-logging'
exclude module: 'xml-apis'
test {
exclude module: 'groovy-all'
compile {
exclude module: 'hibernate'

dependencies {
compile( "org.grails:grails-crud:$grails.grailsVersion",

bootstrap "org.grails:grails-plugin-tomcat:$grails.grailsVersion"