Rewriting Perl Code for Raku III: The Sorceror

Last week, we started testing, learned how to create proper Raku classes, and the basics of functions. This time we’ll take a closer look at functions, arguments, and make some decisions about the API. And maybe while writing this I’ll argue myself out of a decision. It’s happened before.

One good thing about writing about a module is that you can slip into a certain mindset. For instance, right now I’m thinking a few paragraphs ahead, wondering how to explain why I changed the API from Perl 5 references to regular Raku types.

It’s at odds with some of the principles I laid down at the start, which states that I should have minimal changes in the API from Perl to Raku. In Perl 5, you would create the “filesystem root” object like so:

my $root = OLE::Storage_Lite::PPS::Root->new(
  [ 0, 0, 0, 25, 1, 100 ],
  [ 0, 0, 0, 25, 1, 100 ],
  [ $workbook, $page_1, $sheet_1 ]
);

with a bunch of references to lists. By all rights, and the principles I set up earlier, the Raku equivalent should be almost exactly the same:

my $root = OLE::Storage_Lite::PPS::Root.new(
  [ 0, 0, 0, 25, 1, 100 ],
  [ 0, 0, 0, 25, 1, 100 ],
  [ $workbook, $page_1, $sheet_1 ]
);

In fact, all I did was copy and change two characters, specifically the Perl ‘->’ to the Raku ‘.’ operator. Clean, and very simple. And I think what I’ll do is actually just change the code back to using the Perl reference, at least in the API. Dereferencing it will be just a few lines, and I’ll have to change it in the tests as well, but I think the pain will be worthwhile.

This way I don’t have to field questions like “Why did you end up potentially breaking old code?” during talks. See, speaking at conferences about your code really can be a useful motivator!

I’d like a formal argument, please

So, I think I’ve settled on Perl-style formal references, at least for the current iteration. There are actually better ways to do this, but I’ll leave that for the proper Raku version. For right now, quick-n-dirty is the name of the game.

Moving on, we see an important method in the original Perl code, saving an object to disk.

sub save($$;$$) {
  my($oThis, $sFile, $bNoAs, $rhInfo) = @_;
  #0.Initial Setting for saving
  $rhInfo = {} unless($rhInfo);
  # ..
}

As I’ve mentioned before, OLE::Storage_Lite has been around for a long, long time. And it’s obvious here. Function prototypes (not signatures, which are a different kettle of fish) and the use of ‘$oThis’ instead of the more conventional ‘$self’.

Being prototypical

Prototypes were originally meant as a way to save you from having to write checks in your code. Theoretically, if your function was called sub save($$) and you tried to call it with save($fh) you would get an error, because the ‘$$’ means the subroutine took two arguments, and you gave it just one.

But it also predated objects (yes, Virginia, objects in Perl haven’t been around all that long.) and they could have unforeseen side effects. So they were a fad for a while, but quickly faded out of existence.

These days they’re a reason for a more experienced Perl hacker to take the junior aside and explain quietly why we don’t use those anymore, and point them to some modern references, like Modern Perl (not an affiliate link, yet.)

Let’s at least partially convert that to Raku, like so:

method save($sFile, $bNoAs, $rhInfo) {
  #0.Initial Setting for saving
  $rhInfo = {} unless($rhInfo);
  # ..
}

The ‘$oThis’ means that this is a method call, so instead of writing sub save( $oThis, ... ) we can rewrite it to a method and gain ‘self’ instead of the arbitrary variable ‘$oThis’. Of course we do have to do a search-and-replace on ‘$oThis’ with ‘self’, but that’s relatively simple. More complex is what to do with the ‘;’ in the original prototype.

Having options

It’s worth pointing out that OLE::Storage_Lite is taken at least in part from another (larger) module, OLE::Storage. This means that the internal code is redundant in a few places. Raku would let us rewrite what we have as:

method save($sFile, $bNoAs, $rhInfo = {}) {
  #0.Initial Setting for saving
  # ..
}

making $rhInfo an optional variable with a default value. Now, this is a pretty common pattern for a recursive method, so I did a bit of digging. Namely I grep’ed for ‘save’ in the original (all-in-one) Storage_Lite.pm module, and found no recursive calls to it.

Debugging both sides now

This is also where the test suite I wrote earlier comes in handy, as it actually exercises the ‘save’ method. So I added a quick debugging message warn "Saving $rhInfo"; to my local copy of the code, and ran the test suite. Seeing just one ‘Saving …’ message in my test output convinced me it wasn’t recursive. So now the code just looks like:

method save($sFile, $bNoAs) {
  #0.Initial Setting for saving
  my %hInfo;
  # ..
}

Also, since $rhInfo is created in this method, there’s no reason to leave it as a reference. So the initial ‘r’ goes away, and we have left just ‘%hInfo’. It may get passed in to other methods, but Raku lets us pass hashes and arrays as ordinary variable types, so I’ll take advantage of that.

To be fair, leaving it as a reference would have saved me a bit of typing, but I’d already kind of decided that at least internally I’d try to use Raku types and calling conventions, and that left me with the choice of how to pass variables around.

Having options

Finally, there’s the question of what to do with the semicolon. Remember at the start, the function prototype was ‘($$;$$)’ which meant $oThis and $sFile were before the semicolon, and $bData and $rhInfo were after. I can now reveal that ‘;’ in a Perl prototype means that whatever appears afterward is optional.

True to Raku’s nature, I can account for this in at least two ways. One way would be to decide that $bData is always there and just has a default value, probably 0. That would look like method save( $sFile, $bData = 0 ). But the documentation puts $bData in square brackets, indicating that it’s optional.

Raku has an alternate syntax to indicate if a variable is optional, which looks like method save( $sFile, $bData? ). I think this method is better than the alternative syntax because it states clearly that $bData is optional. Both methods work, I just happen to like the ‘?’ modifier.

Waiting for Huffman

Moving on, we have this wonderful line of code:

$rhInfo->{_BIG_BLOCK_SIZE}  = 2**
              (($rhInfo->{_BIG_BLOCK_SIZE})?
                  _adjust2($rhInfo->{_BIG_BLOCK_SIZE})  : 9);

When I was translating this initially, I was in something of a drone mindset, not truly thinking about what I was doing. I’d copied the $rhInfo variable into the method signature and just kept on writing. I ended up with a statement that I eventually shortened quite a bit.

$rhInfo.<_BIG_BLOCK_SIZE> = 2**
  ( $rhInfo.<_BIG_BLOCK_SIZE> ??
    _adjust2( $rhInfo.<_BIG_BLOCK_SIZE> ) !!
                                        9 );

The ‘.’ after $rhInfo indicates we’re dealing with a reference, and the <..> notation is now how barewords look inside hashes. The old {_BIG_BLOCK_SIZE} is still there, but it’s pronounced {‘_BIG_BLOCK_SIZE’}. A lot of people use the {‘..’} in Perl already so it’s not a big change, and it actually simplifies the backend enormously.

Also, at the start Larry and Damian pulled statistics on Perl code from CPAN and other repositories. They were looking for operator frequencies, among other things. Frequently used operators like qw() and -> got even shorter in Raku.

Others, like the ternary operator, weren’t so lucky. It got longer, and stretched to ‘?? .. !!’. So this is one place where the code will look a little funky. Maybe one day I’ll write a slang to fix it, but back to work.

Trimming the verge

Earlier I mentioned that this module was trimmed down from a much larger full OLE reader/writer. This was the first place that became evident. Since $rhInfo is now called %hInfo and initialized inside the method, this statement deserves to be looked at a little closer.

my %hInfo;
%hInfo<_BIG_BLOCK_SIZE> = 2**
  ( %hInfo<_BIG_BLOCK_SIZE> ??
    _adjust2( %hInfo<_BIG_BLOCK_SIZE> ) !!
                                        9 );

After replacing $rhInfo with %hInfo this is what I got. But since %hInfo is defined just above, the test %hInfo<_BIG_BLOCK_SIZE> will never be true, so this entire block can be reduced to:

my %hInfo = _BIG_BLOCK_SIZE => 2**9;

While I’m here I’ll delete _adjust2(). No code pathway uses it, so out it goes. I’ll restore it if I have to, but right now I want the test scripts to pass, and that’s it. I’ve got the original source, and a map from Perl to Raku, and that’s all I need.

Culling yaks from the herd

Where there’s smoke there’s fire, so I stop what I’m doing and grep out every ‘sub X’ call in the source, putting it in a scratch monkey. Then I go through the source (which I have below the new Raku source, deleting lines as I go) and look for methods that aren’t used, like adjust2(). I delete each of these methods with extreme prejudice, because each line of code I don’t see is one I don’t have to translate.

Checkpoint in git, and now it’s time for a lunch break. Afterwards, I’m getting into the save() method, and see what looks like a new yak to shave. Or a package to translate, to be precise.

  if(ref($sFile) eq 'SCALAR') {
    require IO::Scalar;
    my $oIo = new IO::Scalar $sFile, O_WRONLY;
    $rhInfo->{_FILEH_} = $oIo;
    # ...
}

In both Raku and Perl, you can create a single method called new( $sFile ) that treats $sFile as either a filename (scalar), file content (scalar reference) or file handle (scalar object.) In Perl, if we wanted to handle filenames, file contents, or file handles differently, we’d have to switch like this, or have different method names.

In Raku, we can handle this differently. In fact I can write the code to save() to a filename, and add save() to a filehandle later with no modifications needed. Above, I briefly touched on the fact that you can write more than one new() method, as long as the two method signatures were distinct.

multi method save( Str $filename ) {...}
multi method save( IO::Handle $fh ) {...}

Raku will let you write two methods called save(), as long as it can tell which one to call at runtime. So, I can call $root.save( '/tmp/test.xlsx' ); or $root.save( $out_filehandle ); and Raku will “dispatch” it to the right save() method automatically.

We call it ‘multiple dispatch’ for just that reason, dispatching a function call to multiple versions of a method. And this means that I can write the first save( Str $filename ) method without worrying about the other methods. I don’t have to add a new if-then branch to the existing code, or modify save() in any way.

I can just write my save() method and ignore the other IO:: types. Also, if someone gets my code later and wants to add a save() method that saves to something I know nothing about, they can write their new save() method without interfering with mine.

In this installment we’ve covered the basics of function and method calls, delved into the ternary operator, removed dead code and learned a little about multiple dispatch. Next time, we’ll open the binary filehandle we created above and delve into the mysteries of pack() and unpack().

I’ll also show you a new (yes, I couldn’t resist) grammar-based version of pack() that should cover the entire Perl gamut of packed types, with a bit of patience and a large enough test suite.

As always, gentle Reader, thank you for your time and attention. If you have any (constructive, please) comments, criticisms or questions, please let me know in the comment section below.