IdrisDoc: Text.Literate

# Text.Literate

A simple module to process 'literate' documents.

The module uses a lexer to split the document into code blocks,
delineated by user-defined markers, and code lines that are
indicated be a line marker. The lexer returns a document stripped
of non-code elements but preserving the original document's line
count. Column numbering of code lines are not preserved.

The underlying tokeniser is greedy.

Once it identifies a line marker it reads a prettifying space then
consumes until the end of line. Once identifies a starting code
block marker, the lexer will consume input until the next
identifiable end block is encountered. Any other content is
treated as part of the original document.

Thus, the input literate files must be well-formed w.r.t
to code line markers and code blocks.

A further restriction is that literate documents cannot contain
the markers within the document's main text: This will confuse the
lexer.

record LiterateError
  MkLitErr : (line : Int) -> (column : Int) -> (input : String) -> LiterateError
  line : (rec : LiterateError) -> Int
  column : (rec : LiterateError) -> Int
  input : (rec : LiterateError) -> String
record LiterateStyle

  Description of literate styles.

  A 'literate' style comprises of

  - a list of code block deliminators (`deliminators`);
  - a list of code line markers (`line_markers`); and
  - a list of known file extensions `file_extensions`.

  Some example specifications:

  - Bird Style

      MkLitStyle Nil [">", "<"] [".lidr"]

  - Literate Haskell (for LaTeX)

      MkLitStyle [("\\begin{code}", "\\end{code}"),("\\begin{spec}","\\end{spec}")]
      Nil
      ["lhs", "tex"]

  - OrgMode

      MkLitStyle [("#+BEGIN_SRC idris","#+END_SRC"), ("#+COMMENT idris","#+END_COMMENT")]
      ["#+IDRIS:"]
      ["org"]

  - Common Mark

      MkLitStyle [("```idris","```"), ("<!-- idris","--!>")]
      Nil
      [".md", ".idris.md"]

  MkLitStyle : (deliminators : List (String, String)) -> (line_markers : List String) -> (file_extensions : List String) -> LiterateStyle
    deliminators

      The pairs of start and end tags for code blocks.

    line_markers

      Line markers that indicate a line contains code.

    file_extensions

      Recognised file extensions. Not used by the module, but will be
      of use when connecting to code that reads in the original source
      files.

  deliminators : (rec : LiterateStyle) -> List (String, String)

    The pairs of start and end tags for code blocks.

  line_markers : (rec : LiterateStyle) -> List String

    Line markers that indicate a line contains code.

  file_extensions : (rec : LiterateStyle) -> List String

    Recognised file extensions. Not used by the module, but will be
    of use when connecting to code that reads in the original source
    files.

MkLitErr : (line : Int) -> (column : Int) -> (input : String) -> LiterateError
MkLitStyle : (deliminators : List (String, String)) -> (line_markers : List String) -> (file_extensions : List String) -> LiterateStyle
  deliminators

    The pairs of start and end tags for code blocks.

  line_markers

    Line markers that indicate a line contains code.

  file_extensions

    Recognised file extensions. Not used by the module, but will be
    of use when connecting to code that reads in the original source
    files.

extractCode : (specification : LiterateStyle) -> (litStr : String) -> Either LiterateError String

  Given a 'literate specification' extract the code from the
  literate source file (`litStr`) that follows the presented style.

  specification

    The literate specification to use.

  litStr

    The literate source file.

    Returns a `LiterateError` if the literate file contains malformed
    code blocks or code lines.

unlit : (specification : LiterateStyle) -> (litStr : String) -> Either LiterateError String

  Synonym for `extractCode`.