The change of year is a good occasion to look back. Here I want to reflect on the development of Perl 6, its compilers and ecosystem.

At the start of the year, masak's Perl 6 Coding Contest continued from 2010, concluding in the announcement of the winner. I must admit that I still haven't read all the books I won :-)

Specification

2011 was a rather quiet year in terms of spec changes; they were a mixture of responses to compiler writer and user feedback, and some simplifications and cleanups.

Positional parameters used to be allowed to be called by name; this feature is now gone. That both makes the signature binder simpler, and removes accidental dependencies on names that weren't meant to be public. Read the full justification for more background.

A small change that illustrates the cleanup of old, p5-inherited features was the change that made &eval stop catching exceptions. There is really no good reason for it to catch them, except Perl 5 legacy.

say now uses a different stringification than print. The reasoning is that print is aimed at computer-readable output, whereas say is often used for debugging. As an example, undefined values stringify to the empty string (and produce a warning), whereas say calls the .gist method on the object to be said, which produces the type name on undefined values.

An area that has been greatly solidified due to implementation progress is Plain Old Documentation or Pod. Tadeusz Sośnierz' Google Summer of Code project ironed out many wrinkles and inconsistencies, and changed my perception of this part of the spec from "speculative" to "under development".

Rakudo

Rakudo underwent a huge refactoring this year; it is now bootstrapped by a new compiler called "nqp", and uses a new object model (nom).

It allows us to gain speed and memory advantages from gradual typing; for example the mandelbrot fractral generator used to take 18 minutes to run on a machine of mine, and now takes less than 40 seconds. Speedups in other areas are not as big, but there is still much room for improvement in the optimizer.

With the nom branch came support for different object representations. It makes it possible to store object attributes in simple C-like structs, which in turn makes it much easier and more convenient to interoperate with C libraries.

Tadeusz' work on Pod gave Rakudo support for converting Pod to plain text and HTML, and attach documentation objects to routines and other objects.

Rakudo now also has lazy lists, much better role handling, typed exceptions for a few errors, the -n and -p command line options, support for big integers, NFA-based support for proto regexes and improvements to many built-in functions, methods and operators.

Niecza

It is hard to accurately summarize the development of Niecza in a few sentences; instead of listing the many, many new features I should give an impression on how it feels and felt for the user.

At the start of 2011, programming in niecza was a real adventure. Running some random piece of Perl 6 code that worked with Rakudo rarely worked, most of the time it hit a missing built-in, feature or bug.

Now it often just works, and usually much faster than in Rakudo. There are still some missing features, but Stefan O'Rear and his fellow contributors work tirelessly on catching up to Rakudo, and it some areas Niecza is clearly ahead (for example Unicode support in regexes, and longest-token matching).

Since Niecza is implemented on top of the Common Language Runtime (CLR) (which means .NET or mono), it makes it easy to use existing CLR-based libraries. Examples include an interactive fractal generator and a small Tetris game in Perl 6.

Perlito

Perlito aims to be a minimal compiler with multiple backends, which can be used for embedding and experimenting with Perl 6. It had several releases in 2011, and has interesting features like a Javascript backend.

Ecosystem

The presence of two usable compilers (and in the case of Rakudo, two viable but very different branches) has led to many questions about the different compilers. The new Perl 6 Compiler Feature matrix tries to answer the questions about the state of the implemented features in the compilers.

With Panda we now have a module installer that actually works with Rakudo. It still has some lengths to go in terms of stability and feature completeness, but it is fun to work with.

The new Perl 6 Modules page gives an overview of existing Perl 6 modules; we hope to evolve it into a real CPAN equivalent.

Community

This year we had another Perl 6 Advent Calendar, with much positive feedback both from the Perl 6 community and the wider programming community.

We were also happy to welcome several new prolific contributors to the Perl 6 compilers and modules. The atmosphere in the community still feels relaxed, friendly and productive -- I quite enjoy it.

The year ends like it started: with a Perl 6 Coding Contest. This is a good opportunity to dive into Perl 6, provide feedback to compiler writers, and most of all have fun.

Progress on my grant for error message is slow but steady. Since my last report, I've done the following things:

• Merged the nom-exceptions Rakudo branch, so now you can reliably throw Perl 6 objects as exceptions.
• Implemented several error classes and roles in Rakudo
• Started to throw typed errors both from runtime libraries and from inside the compiler
• Hacked the default exception printer Rakudo to be much more flexible, for example you can now write exception classes that supress backtraces from the standard handler.
• Wrote tests for typed run time and compile time errors, and at the same time developed a test function that makes it easy to write such tests.

It's time for a quick review of how far I am along the various deliverables in the original grant proposal.

• D1: Specification. I think the hard work here is done already, what remains to do is finding good default and how to manipulate them (for example, how to generally switch on/off printing of backtraces?).
• D2: Error catalogue, tests: I've not worked on this one too much. The error classes and roles so far mostly served to exercise the implementation; going through the existing errors from the various compilers and formalizing them will be quite a bit of work, but only moderately complicated.
• D3: Implementation, documentation. Like D1, the hard part is mostly done. We can now throw errors from within the compiler actions and from the setting, next up will be the grammar. Then all places where errors are thrown need to be changed to use the new error classes. Again that'll be much work, but easy to do. Documentation is still missing.

All in all I feel I'm well on the way, and most complex decisions have been made.

For a more user oriented view of the new exception system I'd like to point you to my Perl 6 advent calendar post on exceptions.

In Perl 5, if you want to match a regex against a particular string, you write $string =~ $regex.

In the design process of Perl 6, people have realized that you cannot only match against regexes, but lots of other things can act as patterns too: types (checking type conformance), numbers, strings, junctions (composites of values), subroutine signatures and so on. So smart matching was born, and it's now written as $topic ~~ $pattern. Being a general comparison mechanism is the first function of the smart match operator.

But behold, there were problems. One of them was the perceived need for special syntactic forms on the right hand side of the smart match operator to cover some cases. Those were limited and hard to implement. There was also the fact that now we had two different ways to invoke regexes: smart matching, and direct invocation as m/.../, which matches against the topic variable $_. That wasn't really a problem as such, but it was an indicator of design smell.

And that's where the second function of the smart match operator originated: topicalization. Previously, $a ~~ $b mostly turned into a method call, $b.ACCEPTS($a). The new idea was to set the topic variable to $a in a small scope, which allowed many special cases to go away. It also nicely unified with given $topic { when $matcher { ... } }, which was already specified as being a topicalizer.

In the new model, MATCH ~~ PAT becomes something like do { $_ = MATCH; PAT.ACCEPTS($_) } -- which means that if MATCH accesses $_, it automatically does what the user wants.

Awesomeness reigned, and it worked out great.

Until the compiler writers actually started to implement a few more cases of regex matching. The first thing we noticed was that if $str ~~ $regex { ... } behaved quite unexpectedly. What happend was that $_ got set to $str, the match was conducted and returned a Match object. And then called $match.ACCEPTS($str), which failed. A quick hack around that was to modify Match.ACCEPTS to always return the invocant (ie the Match on which it was called), but of course that was only a stop gap solution.

The reason it doesn't work for other, more involved cases of regex invocations is that they don't fit into the "does $a match $b?" schema. Two examples:

# :g for "global", all matches
my @matches = $str ~~ m:g/pattern/; 

if $str ~~ s/pattern/substitution/ { ... }

People expect those to work. But global matching of a regex isn't a simple conformance check, and that is reflected in the return value: a list. So should we special-cases smart-matching against a list, just because we can't get global matching to work in smart-matching otherwise? (People have also proposed to return a kind of aggregate Match object instead of a list; that comes with the problem that Match objects aren't lazy, but lists are. You could "solve" that with a LazyMatch type; watch the pattern of workarounds unfold...)

A substitution is also not a simple matching operation. In Perl 5, a s/// returns the number of successful substitutions. In Perl 6, that wouldn't work with the current setup of the smart match operator, where it would then smart-match the string against the returned number of matches.

So to summarize, the smart match operator has three functions: comparing values to patterns, topicalization, and conducting regex matches.

These three functions are distinct enough to start to interact in weird ways, which limits the flexibility in choice of return values from regex matches and substitutions.

I don't know what the best way forward is. Maybe it is to reintroduce a dedicated operator for regex matching, which seems to be the main feature with which topicalization interacts badly. Maybe there are other good ideas out there. If so, I'd love to hear about them.

Progress on my grant for error message is slower than expected, as expected :-). Yes, you've read that sentence before.

In the past months, general hacking on the nom branch of Rakudo was just too much fun -- and partially a prerequisite for the exceptions work.

I did manage to redo the backtraces that are generated from error messages.

Backtraces are now generated mostly in Perl 6 code, making them much more hackable. There's a Backtrace class, which is a list of Backtrace::Frame objects, each knowing the code object associated with it, as well as line number and file. (This is both specced and works in Rakudo)

Routines can have the is hidden_from_backtrace trait, which makes them not show up in the default backtrace stringification (one can still request a .full string representation). This is useful for routines which are internally used to generate exceptions, like die().

Rakudo also has a --ll-exceptions command line option which provides PIR-level backtraces, in the rare case the Perl 6 level backtraces hide too much information.

I've also started the nom-exceptions branch in Rakudo, which aims at lifting current limitations in Rakudo's exception handling. Currently die() and friends generate a parrot exception, and then there's a routine that fills the error variable $!. This routine generates a new Exception object, and sticks the parrot exception into it.

This practice means that if you create a subclass of Exception, instantiate it and throw it, you still only get an Exception in the error handler, not an object of the subclass. Since the actual exception type is very important for the ongoing work, that has to change. The branch mentioned earlier allows one to generate a Perl 6 exception, and pass that on as the payload of the parrot exception, which is then unwrapped when filling $!.

As a proof of concept this works, but it suffers from not being robust enough -- as it is, we could accidentally unwrap the payload of a CONTROL exception, placing meaningless junk into $!. So this needs a bit more work, which I plan to do this week (or next, if it proves to be more difficult than anticipated).

As always, your feedback is very welcome.

Rakudo, a popular Perl 6 compiler, is built on top of a smaller compiler called "NQP", short for Not Quite Perl.

Reading through a recent ramble by chromatic, I felt like he said "Rakudo needs NQP to be able to ditch Parrot, once NQP runs on a different platform" (NQP is the "another layer", which sits between Rakudo and Parrot, mentioned in the next-to-final paragraph).

I'm sure chromatic knows that VM independence is the least important reason for having NQP at all, but the casual reader might not, so let me explain the real importance of NQP for Rakudo here.

The short version is just a single word: bootstrapping.

The longer version is that large parts of Rakudo are written in Perl 6 itself (or a subset thereof), and something is needed to break the circularity.

In particular the base of the compiler is written in a subset of Perl 6, and NQP compiles those parts to bytecode, which can then compile the rest of the compiler.

This is not just because we have a fancy for Perl 6, and thus want to write as much of the code in Perl 6, but there are solid technical reasons for writing the compiler in Perl 6.

In Perl 6, the boundary between run time and compile time is blurred, as well as the boundary between the compiler, the run time library and user-space code. For example you alter the grammar with which your source code is parsed, by injecting your own grammar rules.

"Your own grammar rules" above refers to user-space code, while the grammar that is being altered is part of the compiler. If we had written the compiler in something else than Perl 6 (for example Java), it would be horribly difficult to inject user-space Perl 6 code into compiled code from a different language.

And the code not only needs to be injected, but the data passed back and forth between the compiler and the user space need to be Perl 6 objects, so all important data structures in the compiler need to be Perl 6 based anyway.

And it's not just for grammar modifications: At its heart, Perl 6 is an object oriented language. When the compiler sees a class definition, it translates them to a series of method calls on the metaobject, which again needs to be a Perl 6 object, otherwise it wouldn't be easily usable and extensible from the user space.

Now you might think that grammar modifications and changes to the Metaobject are pretty obscure features, and you could get along just fine with an incomplete Perl 6 compiler that neglected those two areas. But even then you'd have lots of interactions between run time and compile time. For example consider a numeric literal like 42. Obviously that needs to be constructed of type Int. What's less obvious is that it needs to be constructed to be of type Int at compile time already, because Perl 6 code can run interleaved with the compilation. So the compiler needs to be able to handle Perl 6 objects in all their generality, which is a huge pain if the compiler is not written in Perl 6.

Rakudo has cheated on that front in the past, and consequently has had lots of bugs and limitations due to non-Perl 6 objects leaking out at unexpected ends. If you ever got a "Null PMC Access" from Rakudo, you know what I mean.

The lesson we learned was that you need a Perl 6 compiler to implement a Perl 6 compiler, even if that first Perl 6 compiler can handle only a rather limited subset of Perl 6.

And there are also quite some benefits to this approach. For example NQP's new regex engine is implemented as a role in NQP. It is mixed into an NQP class which allows us to build Rakudo, but it is also mixed in a Perl 6 class, which allows the generation of Perl 6-level Match objects without any need to create NQP-level match objects first, and then wrap them in Perl 6 Match objects.

That's what NQP does for us. It allows us to actually write a Perl 6 compiler.

We now have a nice table that tells you which Perl 6 compiler implements what..

Such a thing was long overdue. When the topic came up in the past, people have suggested mostly automated solutions that compared test coverage of compiles to generate such a table. Nothing came out of it, it would have been a rather large endeavor. Now Eevee blogged about the lack of some easy overview that tells you what is implemented in Rakudo., and I thought it was time to tackle the problem.

Instead of some advanced automated system, we now have a simple text file, and a short perl script that converts it to a HTML page.

I'd like to thanks Will Coleda, Patrick Michaud and Stefan O'Rear for their contributions, and encourage everybody to keep the data up to date.

Nearly one year ago, the Rakudo Perl 6 developers proudly released the first Rakudo Star, a distribution aimed at showing the world what Perl 6 can look like, and in turn get feedback from more early adaptors.

And feedback we got. While the overall response was very positive, people had one main concern: it was too slow. That didn't come as a surprise, considering that we had focused on features first. Now it was time to change that, and work on massive performance improvements.

That is easier said than done. One of the reasons is that Rakudo is tightly coupled to the parrot virtual machine, but there is a lot of mismatch between the two. For example parrot provides multi dispatch built-in, but not quite with the semantics that Perl 6 needs. Same for parameter binding, objects and a number of other areas.

In the following year, parrot got a new, faster garbage collector, and Jonathan Worthington came up with a cache for type checks at routine call time.

This sped up this simple mandelbrot fractal generator at size 201x201 from 18 minutes to 16 minutes 14 seconds. Actually the speedup was better than that, but we paid a performance penalty for new features, bug fixes and parrot performance regressions.

But it was clear that more substantial improvements where needed. One of the most promising candidates for speedups is a complete redesign of the object model, resulting in the "nom" (new object model) branch of Rakudo. Additionally to providing much more well suited OO primitives than parrot can offer right now, it also allows to share more information between compile time and run time, making a lot of optimizations possible.

Yesterday I sped up some operations on Complex numbers, and implemented a built-in that was missing to run the mandelbrot script. And today I timed it: 3 Minutes. From originally 18 Minutes.

Now that's a speedup by more than a factor of 5. I'm not sure if it will extend to other operators, but it sure is encouraging.

And that's without the optimizations that will now be possible, for example inlining operators. So after a literally slow start, Rakudo Perl 6 has a bright and fast future ahead. And it's already here, just not evenly distributed.

Progress on my grant for error message is slower than expected, as expected :-). Life and work, though rewarding, take their toll. Also many of my original ideas turned out to be not very good, and got shot down by @Larry (with good reason).

However I did get around to lay out some specification for exceptions in S32::Exception (rendered) - some basic roles, and the relationships between exception, Failure (lazy/unthrown exceptions) and backtraces.

More importantly I think I'm at a point where I could start to actually implement some of that stuff, and use that experience to update the specification.

My Hague Grant proposal for designing, implementing and testing structured error messages for Perl 6 has been acceepted, and I've started my work on it in my copious free time.

Before the grant started I've unified the error messages of several compilers to use "Cannot" instead of a wild mixture of "Cannot", "Can not" and "Can't".

In the past week I created a repository for the initial work on the error message spec, and added a list of existing error messages across different compilers, and some notes regarding the upcoming spec.

So far I've outlined some thoughts about separation of concerns, classification of the error messages, testing error messages for certain properties, and calling syntax for die() and fail().

Any constructive feedback on it is very welcome.

Thanks go to Ian Hague and The Perl Foundation for supporting my work financially.

Lately real life has prevented me from blogging, so here are just a few random notes from the Perl 6 developers:

The Perl bug tracker now has tags testneeded and testcommitted, which can mark tests that need or have tests in the spectest suite. Since the URLs for querying these tags are unwieldy and non-obvious, I've created some aliases: http://rakudo.de/testneeded and http://rakudo.de/testcommitted.

Development of the new nqp and rakudo-on-the-new-object-model is progressing nicely. I had some fun porting some PIR code to NQP, and writing some new code. Most interesting to read is the source of the new meta model, much of which is written in a subset of Perl 6 (so quite readable, if you happen to know Perl 6. For example you can see how the method resolution order for multiple inheritance is calculated.

There is a parrot branch that adds a generation garbage collector to parrot. Its release is planned for shortly after the 3.1.0 release due tomorrow. Initial benchmarks show that Rakudo is between 25% and 30% faster on that parrot, as measured by a spectest run. I very much look forward to having that in the parrot main line.

Writing code for niecza is quite a nice experience. It still has a big startup cost, but then runs much faster than rakudo (at least it feels that way). There are still lots of features missing (for example non-integer number literals), but feature requests are usually implement quite quickly.

The Perl 6 developers communicate a lot through IRC. Some of the conversation is still valuable later on, so we have public IRC logs.

The software powering these logs was written especially for #perl6, but works fine for other channels too. Among the other users are TestML, CDK (Chemistry Development Kit), darcs, mojo, Padre, the Perl IDE, Parrot and Rosetta Code.

If you are also developing software, and would like public logs for your channel (either on freenode or irc.perl.org; other servers might be added on demand), feel free to contact me (moritz on freenode, or per email: moritz at faui2k3.org)

Features include: linking to individual lines, permanent URLs and volatile URLs for the current day, automatic linking of URLs and readable color schemes.

A current limitation is that you can't have two channels with the same name from different networks, in case of conflict "first come, first served" holds.

Masak's Perl 6 Coding Contest (short p6cc) is now in its final stage - the public commentary of solutions. So far masak has commented on p1, p2, and p3. I enjoyed the reviews and explanations so far, and look forward to more (except to the p4 review, because I botched the solution to this one).

The reviews made a good read, and here are only a few very minor points that I find worth mentioning. No criticism intended (neither to the author nor the reviewer).

• Matthias' p1 solution uses side effects in subroutine named to-string. I would avoid that in "production" code (for whatever I might mean with that word...), since it's not what I would expect from the name. Instead of a counter, the array indexes could be used to identify which matrix to stringify.
• Masak's review of fox' p1 solution contains the question I wonder why the @items array deserved a plural but the @matrix only a singular.... I'd say that's because @matrix holds one matrix, but @items holds many items. Speaking of which, I don't like the name @matrix - it describe a structure rather than the contents. The structure becomes pretty obvious through the access, but what is in the matrix?
• After viewing my own p1 submission without syntax hilighting, I wonder why I chose so many double blank lines. My vim color scheme uses a dark blue for comments, which means that comment blocks need more visual distance from code, in my personal opinion. But when sharing code, I shouldn't make layout decisions based on non-shared syntax hilighting.
• colomon's p2 submission uses complex numbers for coordinates. I considered that myself, and there's nothing wrong with it. Just strange that he then reinvented subtraction of two complex numbers in sub lines-intersect. Maybe he didn't want to come up with names for the intermediate results, $slope and $axes-intercept might have been viable ones.
• Matthias could have simplified comb: /<&number>/ to just comb: &number -- a regex is just a callable, and when it's explicit in the current scope, you can just use a hard reference to it.
• colomon's p3 solution could, as far as I can tell, replace all regexes with tokens - less backtracking, fewer surprises.
• Reviewing my own p3 solution, masak asked I wonder what stopped moritz from, rather than doing $_ && .including on line 39, doing .?including instead. Either he considered that too cute, or he didn't consider it. I did consider it, but I decided against it, because the two things are subtly different. Mine only calls the method if the topic is true, whereas .?including always tries the call - even on a type object, where accessing an attribute leads to a fatal error. I think in my code that case doesn't show up, but it did appear during debugging in earlier versions.

2010 has been a busy year for the Perl 6 developers, and came with a noticeable distribution release, many new modules and much fun for the people involved. Here's a short, subjective reflection of this year's Perl 6 events.

Specification

While some specification changes had substantial impact on the compiler writers (and were usually in turn triggered by their worries), the user mostly saw maturing of experimental features, smoothed APIs and some few new features.

Lists, lists, lists

Just as last year, there were a lot of discussions on how lists and related types worked. Much of it was driven by the efforts to implement proper lazy lists in Rakudo.

The result is a much more solid list model, which uses immutable iterators under the hood - a fact that is hidden quite well from the user.

The sequence operator (previously known as "series operator") is a powerful tool for creating lazy lists. It has been extensively refactored to solve problems both with its implementation and usage. It now takes an optional closure left of the ... operator, and a limit that terminates the sequence if it matches true:

# Lazy list of Fibonacci numbers up to (but excluding) 100:
my @fib := 1, 2, *+* ...^ * >= 100;

Date and Time

After several iterations, excessive bikeshedding and serious hacking, Perl 6 now has built-in classes for handling times and dates. They are inspired by the DateTime and Date::Simple Perl 5 modules. The biggest difference is probably that DateTime objects are immutable in Perl 6.

This part of the specification is implemented completely in Rakudo.

Zip meta operator

The Z zip operator can now also act as a meta operator. Thus an easy way to add two lists pairwise and lazily is now

my @sum = @list1 Z+ @list2;

Other changes

• The .pick method, which randomly takes one or more elements from lists and hashes, has been split up into two: @a.pick(3) returns 3 distinct, random items from array @a, while @a.roll(3) does three independent random choices, resulting in possible duplicates in the result list.
• The scoping of lexical multi routines and their protos has been clarified and overhauled (see this discussion of what was wrong previously, and the resolution).
• The numeric roles, buffers and Whatever type have received significant updates

Community

In 2010 we had a remarkable influx of friendly, interested, skilled and enthused newcomers to the Perl 6 community. This is the result of increased marketing outside the Perl community, well publicized releases, great technology and a friendly community.

Community expansion

Two challenges or contests have been announced this year.

Moritz Lenz published a series of "weekly" challenges, guided tasks to implement something that the Perl 6 community needs: A website for the ecosystem, a feature in a compiler and other small things that could be tackled without much prior knowledge.

The overall response was very good, and several people used it as a quick start into the Perl 6 community, and stayed.

Towards the end of the year, Carl Mäsak announced his Perl 6 coding contest. The submitter with the best solutions to five well known programming tasks is to win 100€ worth of books.

Another far-reaching project was the Perl 6 advent calendar for 2010, which attracted more than forty thousand visitors .

In an attempt to make Perl 6 compilers easier available to the masses, John Harrison implemented a web frontend to the Rakudo REPL and made it available at try.rakudo.org.

Conferences

There was a big Perl 6 hackathon (though we had more discussions than hacking) at YAPC::EU 2010 in Pisa. Many Perl 6 contributors, compiler writers and users met and discussed pressing topics in the realms of specification, implementation roadmap, measuring progress and community management. See the meeting notes for details.

Of course there were also some Perl 6 talks at YAPC::EU, many of which seemed well received by the audience.

Perl 6 talks were also held at the Netherlands Perl Workshop, YAPC::Russia, Norwegian Unix User Group and OSDC France, as well as many other conferences which the author forgot :-).

Repository changes

Due to neglected maintenance, the Pugs repository had to be shut down. It has been migrated to git, and split up into several repositories under the perl6 organization on github. Notable parts include:

roast

the Perl 6 test suite

specs

the specification

perl6.org

the main Perl 6 website

modules.perl6.org

the Perl 6 modules website

ecosystem

the module list repository

mu

the remnants of the old pugs repository

While the transition was mostly ad-hoc and not really planned for, most of the resulting confusion could be resolved fairly quickly.

Module ecosystem

While we still lack a proper module distribution system, we now have a website of known Perl 6 modules and a module installer.

But most importantly the number of modules and module authors is steadily increasing (82 known Perl 6 modules at the time of writing, compared to 45 last year). While we still lack the wealth of the Perl 5 ecosystem, there are now database modules, HTTP client and server modules, serialization, file handling tools and so on.

Implementations

Rakudo

Most importantly, this year saw the first Rakudo Star release. Rakudo star is a distribution of the Rakudo compiler, modules and documentation. While it is still a kind of preview release, some few production usages of the Rakudo Perl 6 compiler and distribution have been spotted in the wake of this release.

Also a good part of the Rakudo code based has been replaced during a major refactor, which bases Rakudo on top of a new grammar engine.

Major improvements to the compiler include

• an implementation of lazy lists
• lexical classes and roles
• Perl 6 level stack traces
• much more solid meta object model, which allows the user to create and modify classes programmatically at run time
• implementation of the s/search/replace/ and s[search] = replace() syntactic features, along with several new regex adverbs and variable interpolation into regexes
• improved interpolation in double-quoted strings: array and hash variables now properly interpolate when the expression ends in a bracketing construct
• an improved read-evaluation-print loop, which now remembers variables from previous lines, and also automatically prints the result if no output was produced
• multi level array and hash autovivification
• binding and read-only binding of variables
• a solid implementation of the DateTime and Date classes
• MAIN and USAGE subroutines
• the magic $*ARGFILES file handle and get (comparable to while (<>) { ... } in Perl 5)
• an implementation of basic feed operators

During YAPC::EU the Rakudo contributors decided to target multiple virtual machines: besides the current parrot backend we want to support at least the CLR (.NET).

With this goal in mind, and the need for major performance improvements, Jonathan Worthington prototyped a new, efficient meta object model for parrot in C#, and used that as a base for the new CLR backend. He got help from Matthew Wilson, and Martin Berends started porting the effort to the JVM. Jonathan explained some of his work nicely on the 6guts blog.

In 2011 we will likely see a port of the meta object implementation to parrot, and the beginnings of a Rakudo port to the CLR and JVM.

Niecza

In June, Stefan O'Rear started taking notes on how to compile Perl 6 to the Common Language Runtime (CLR). In November he announced the Niecza Perl 6 compiler, focused on the generation of efficient code.

It already has an impressive list of features, including proper Longest Token Matching, a feature of regexes and grammars that no other Perl 6 compiler has implemented so far.

Summary

2010 was a very rewarding year for the Perl 6 community. With Rakudo there was a compiler available, with which small and medium scale projects can be fun to write. Niecza is quickly catching up.

People experiment with Perl 6, join the community and bring fresh ideas. There is still a long road ahead of us, but the author feels that this road is getting broader and more accessible with each step.

Usually I use Perl 6 as my programming language of choice. But about a week ago I started to plan and code an application for which the richness of the Perl 5 ecosystem outmatch the design weaknesses of Perl 5, compared to Perl 6.

I have quite a bit of Perl 5 experience too, but so far I mostly used it for smallish tasks, and didn't have too much use for many frameworks.

Here is a list of some tools I used, and my experience with them. I hope that my experiences can help the Perl 5 community to improve some edges here and there, and serve as an inspiration for Perl 6 modules authors what problems to approach.

cpanminus - painless module installation.

cpanminus gives easy access to the wealth of the CPAN. The module installation worked perfectly without any configuration or interaction.

Just when a XML parser had a dependency on an external C header file (expat.h), a module installation failed. Since Perl modules don't have a way to declare external dependencies, that's probably as good as a module installer can work. Kudos! The only improvement I can think of is directly showing the error message, instead of having it to dig up from a log file. I don't know if there's an easy way to automate that though.

DBIx::Class for the database

I store data in a postgresql database. From lowest to highest abstraction, the Perl modules involved are DBD::Pg (the database driver), DBI (the driver-independent database interface) and DBIx::Class, an object relational mapper.

Postgres itself is fantastic, and DBD::Pg and DBI look rock solid to me. I've worked with DBIx::Class before, and liked it. Upon rereading the documentation, I found that since my last usage the recommended usage pattern has changed. Writing result classes into the Result::MyClass namespace and result set into ResultSet::MyClass has made the result sets more accessible. Since they are a key feature of DBIx-Class, I welcome this change, and adopted it very naturally.

A small left-over from the previous scheme made the transition a tad harder than it had to, but upon reporting it on the #dbix-class IRC channel (on irc.perl.org), I immediately got commit access, and fixed it in the source.

Since I deal with trees, I was happy to discover a DBIx::Class plugin for nested sets. I was less happy to discover that it broke basic object creation, and had a bug that prevented merging of trees, a feature advertised in the documentation. Luckily both were very easy to patch, the patches now live in the bug tracker. I hope the maintainer applies them soon.

The nested set extension comes with a good test suite, but it seems it hasn't had much real world usage. I think that with some more usage (and maybe a few more bug fixes), it'll turn into a very good module.

Mojolicious for the web frontend

While waiting for the Catalyst dependencies to install, I decided on a whim to try out Mojolicious, a new-ish web framework. Or more precisely Mojolicious::Lite, a simplified API that lets you keep the whole application in a single file.

Now there were a lot of small, rough spots in Mojolicious. The community on IRC was very helpful, and asked me to record my findings on a wiki page - which I did.

What really bugs me about Mojolicious is that the built-in template system produces very incomprehensible error messages. It uses a mixture of verbatim text and perl code, separated by tags with various semantics (for example tags that just execute code, those that execute and insert the result, optionally with HTML/XML escaping).

Unfortunately that means that the code you write differs from the code that perl executes, which makes the error messages pretty useless.

My first suggestion to improve that situation is to display the generated code in the error message, in addition to the template code (and make the generated code as simple as possible.

If the generated code is non-trivial, it would help to add some markup to distinguish the user-written code from the code that the generator adds around it. I have no idea how easy or hard that would be to implement, though.

Conclusions

The Perl 5 modules were mostly very easy to use, and the corresponding communities very attentive and helpful.

If there's something the authors can learn from Perl 6, then it's a love for better error messages.

The Perl 6 world can aspire for such a rich and easy-to-use module system.

tl;dr;; version: arbitrary API limitations do more harm than good, even if meant well in the first place.

Most advanced libraries that help you with date calculations have separate data types for a point in time, and a time span. That's because those two concepts actually have different semantics. It doesn't make sense to add two points in time, but it does make sense to add two durations, or add a duration to a point in time.

In Perl 6, those two types are called Instant and Duration. And obviously it makes sense to multiply a Duration with a number, but it doesn't make sense to multiply two Durations, or take a power of a Duration. Right?

That's the opinion that both the Perl 6 specification and the implementation in Rakudo reflected. Until recently, when somebody started to actually use it.

That's when the real world struck back. Carl Mäsak did some timings, and then calculated averages and standard deviations. And for calculating standard deviations, you actually have to square those durations, add them up, and then calculate the square root.

So this perfectly legitimate use case shows that multiplication (and also exponentiation) are perfectly fine operations on Durations. Likewise the current specification disallows division of two Durations. Why? It's perfectly fine to ask for the ratio of two time spans. How much longer (or shorter) are my meetings with my current boss, compared to those with my previous boss? That's the question that Duration / Duration answers.

So, the real world taught me that putting restrictions on the allowed operations is a bad idea. It was meant well, it was supposed to catch operations that don't made sense to the designer, and presumably would catch some error that a confused beginner might make. But in the end it did more harm than good.

Currently the Duration class stores a number, and re-dispatches all operations to that number, forbidding some of them. Having learned my lesson, I suggest we get rid of it, and have Instant - Instant return that number straight away. If some day we want to add functionality to Duration, we can still create that class as a subclass of the number.