Patterns of Domain Specific Languages

From DSLs

Lately, involved in several discussions related to the subject, I noticed a global misunderstanding and an obvious blur about the Domain Specific Languages. This resulted in unbelief, generating a tangible doubt about its presumable use in a realistic context of nowadays industry.

Some of the blame goes to people like me, who hastily expose its ideal form, making it look like an unrealistic payoff for a lot of efforts. Such mistakes have been done frequently, particularly in IT field, recently in SOA, turning it into sort of imaginary! Other less recent examples exist, like Client-Server technology that was to allow users to draw their interfaces…

Arguments of some respectful people in today’s industry had corrected my view, and made me focus on the core principle, letting the ultimate come naturally, as the appeal calls to it due to maturity.

Linking to examples from the real world can clarify some points here. I like an example of Erik Meijer in QCon 2007 London, stressing that software is like buildings. We don’t build our own house ourselves. Rather we come in on later phases for decorating and arranging it. And in everyday life we keep configuring it, changing lights, paint color, carpets, maintaining and tuning it using simpler tools adapted to our needs. I’ll cross this with a quote of Ted Neward “VB enabled managers to write programs. The problem is they really did! ”. So the purpose, at least in the short term, is not to magically make business people write their own applications. But rather turning the software into softer, more flexible product, which can be bent and adjusted to shape the new ever-changing business needs, pursuing the ever-changing market.

DSLs are barely new, they always existed responding to the natural need of providing tools to humans for configuring their software. Their existence was indeed minimal, focused on satisfying these urgent needs of the client’s business. Examples of common DSLs are xml config file, Xslt, aspx, fit framework for unit tests, excel, CMSs templates, workflows, wikis, Sql … etc.

Some of these DSLs have become quite mature in their domain, providing a unique tool for a specific purpose. I guess this maturity is due to their convergence with that domain. This convergence results from the fact that they were specifically designed for the domain, or at least found to fit perfectly in it, and reflect not only its vocabulary, but also its structure and interactions.

Given this observation, I would disagree about Xml as a proper wide use DSL. Xml in its own is a General Purpose Language. Proposed solutions, like XSD, DTD and other schemas, don’t offer much more than rules and grammars. A DSL should have an easy understandable syntax that reflects the model. That doesn’t often correspond to xml. One of the reasons can be its verbose syntax (caused by redundant opening and closing tags). Another is its structure that imposes a hierarchical thinking about the problem.

I like thinking of DSL as a language created for the problem, not forcedly adapted to fit the problem. Forcing generates awkwardness and meaningless semantics that break the readability and the coherent side of a DSL.

Nowadays there is a sufficiently wide choice of DSL tools, they come in different flavors (list is not exhaustive):

- Xml based DSLs, meaning generating xml out of personalized graphical or specific purpose syntax. Usually used as a configuration representation, it can’t escape hierarchical structure for long but still handy. - Some General Purpose Languages friendly syntax permits an expression of domain semantics in a pronounced readable manner. We can link to this some design patterns that enhance the ubiquitous language produced by these languages. Like Expression Trees, Specifications, Fluent Interfaces…etc. Examples of such languages are Ruby and Prolog. A limitation of this kind of DSL is the compiler grammars that insist on some differences that are not obvious for humans. - General Purpose Languages with extensible syntax, the possibility of adding keywords and semantics to the language through macros or metadata. Examples include Lisp, F#, Nemerle.net. Such kind of GPL urges discipline, as it might introduce ambiguousness and verbosity. - Compiler-Compiler tools that are tools and APIs for declaring some grammar for a parser that will parse the customized syntax, verifies it against the grammar, then generate something useful. It might be code, config files or something else. Such tools, e.g. SableCC, Antlr, have no influence on the chosen structure of the built languages since we do it from scratch. Still they require much more efforts for validation, error checks and maintainability.

Clearly enough there is no single DSL that satisfies all needs. So, logically enough, tradeoffs should be done.

Now, as choices of DSL tools diverge, their applicability areas vary in different domains, applications and contexts. The aim is to introduce modifiability by identifying an important frequently changed area in the software, to abstract it and then introduce it into a friendly DSL. This modifiability adds flexibility, agility and dynamism.

Other possible advantage of that is a connection between userstories and code, where the user, together with developers, can write her needs and expectations in an appropriate domain language. That language can be interpreted into running code.

I had an interesting discussion with Jim Coplien - he is one of the kindest and most interesting speakers I met at QCon - about TDD. The discussion didn’t mention DSLs at all, but it made me think of them as a possible solution to the absence of trace between userstories and code. (It can be called DSL driven design and development, which maps well to Domain Driven Design of Eric Evans…)

Like any software development practice, DSLs should be applied right and in the right context. This comes with experience (mostly domain experience, but also conceptual and architectural one), experience in finding the right abstraction and then mapping it to the right tool for a given context.

Such an exercise can be hard even for experienced developers and architects. The choice is not evident given the diversity of domains, diversity in the same domain and diversity of DSL tools styles.

Abstraction is the key; it comes from years of experience in identifying common places to extract DSLs. Together with the domain experience that is instrumental to point out the most profitable modifiability and the most appropriate DSL style, it could be translated into templates in form of patterns, Patterns of Domain Specific Languages Use. This way, instead of generalizing solutions, thus falling back into General Purpose Language, we extract best practices to be used for boosting learning and experience curve. Examples of common places where DSLs apply well are Strategies, Expression trees, Specifications, Factories in banking, finance, workflows…

To conclude this article, and to rescue it from the pitfall of excessive abstraction, I will try to demonstrate a scenario where pattern can be identified and abstracted. A whitepaper listing several discovered patterns is the subject of a research that I started with my colleague Julien Delhomme and will be posted on www.domainspecific.org. Updates about the project with code snippets would also be available on a wiki on the same website.