Reader Syntax

siiky

2022/03/15

2022/07/07

en

TIL a bit of reader syntax magic. With very few lines of code I was able to make available the #!sql reader syntax to let me read the contents of SQL files as a literal string (any file actually, but I was thinking of using it for SQL files only).

; This:
#!sql "path/to/file.sql"
; Into this:
"CREATE TABLE entries (\n    cid      TEXT        PRIMARY KEY NOT NULL UNIQUE,\n    name     TEXT        NOT NULL,\n    consumed BOOLEAN     NOT NULL DEFAULT FALSE,\n    url      TEXT        UNIQUE,\n    type     VARCHAR(10) NOT NULL REFERENCES types (name)\n);\n\nCREATE TABLE nodes (\n    id   TEXT        PRIMARY KEY NOT NULL UNIQUE,\n    name VARCHAR(20) UNIQUE\n);\n\nCREATE TABLE pins (\n    node TEXT NOT NULL REFERENCES nodes (id),\n    cid  TEXT NOT NULL REFERENCES entries (cid)\n);\n\nCREATE TABLE types (\n    name VARCHAR(10) PRIMARY KEY NOT NULL UNIQUE\n);\n"

Here's the necessary code in its entirety:

(set-read-syntax!
  'sql
  (lambda (port)
    (let ((path (read port)))
      (unless (string? path)
        (syntax-error "The #!sql syntax expects a string"))

      (let ((sql-stmt (call-with-input-file path (cute read-string #f <>) #:text)))
        (unless (string? sql-stmt)
          (syntax-error "Failed reading the SQL file"))
        sql-stmt))))

There's one caveat with this approach, however: the reader syntax will be available to the whole program, not just the file or module that defined or imported it. This means that the identifiers must be unique, otherwise different definitions will collide with each other and the compiled program won't be what you expect. AND I think that it will be available not only at compile time, but at runtime as well -- very good to keep in mind!

Someone on IRC mentioned that it's possible to use -extend (-X) to make it available at compile time only. As an example, they said that compiling with -X srfi-19-literals would allow one to write #@1-1-22. Try this, after installing SRFI-19:

csi -R srfi-19 -R srfi-19-literals -p "#@`date +'%Y-%m-%d'`"

Relevant CHICKEN documentation for set-read-syntax! & friends:

And the relevant SRFI-19 literals documentation:

-----

To make it more obvious why this is cool, here goes a slightly more realistic, though still simple, example.

Let's say we have these SQL files:

-- schema.sql
CREATE TABLE sometbl (col TINYINT NOT NULL);

-- data.sql
INSERT INTO sometbl (col) VALUES (0),(1),(2),(3),(4),(5);

-- select.sql
SELECT rowid, col FROM sometbl;

An example.scm:

(import sql-de-lite)

(define-constant schema #!sql"schema.sql")
(define-constant data #!sql"data.sql")
(define-constant select #!sql"select.sql")

(print (call-with-database
         'memory
         (lambda (db)
           (let ((schema (sql db schema))
                 (data (sql db data))
                 (select (sql db select)))
             (query fetch-all schema)
             (query fetch-all data)
             (query fetch-all select)))))

And the set-read-syntax! call from before wrapped up in a module sql-reader-syntax. After compiling the module you can compile the example with csc -X sql-reader-syntax example.scm, and this is the result of running it:

$ ./example
((1 0) (2 1) (3 2) (4 3) (5 4) (6 5))

The example.scm is basically transformed into this before being compiled:

(import sql-de-lite)

(define-constant schema "CREATE TABLE sometbl (col TINYINT NOT NULL);")
(define-constant data "INSERT INTO sometbl (col) VALUES (0),(1),(2),(3),(4),(5);")
(define-constant select "SELECT rowid, col FROM sometbl;")

(print (call-with-database
         'memory
         (lambda (db)
           (let ((schema (sql db schema))
                 (data (sql db data))
                 (select (sql db select)))
             (query fetch-all schema)
             (query fetch-all data)
             (query fetch-all select)))))

And notice how schema, data, and select are constants (defined with define-constant, kinda similar to the static keyword in C).