# ic/rodfiles

Theme:

- Index

Search:
Group by:
Source Edit

Low level binary format used by the compiler to store and load various AST and related data.

NB: this is incredibly low level and if you're interested in how the compiler works and less a storage format, you're probably looking for the `ic` or `packed_ast` modules to understand the logical format.

# Overview

`RodFile` represents a Rod File (versioned binary format), and the associated data for common interactions such as IO and error tracking (`RodFileError`). The file format broken up into sections (`RodSection`) and preceded by a header (see: `cookie`). The precise layout, section ordering and data following the section are determined by the user. See `ic.loadRodFile`.

## A basic but "wrong" example of the lifecycle:

1. `create` or `open` - create a new one or open an existing
2. `storeHeader` - header info
3. `storePrim` or `storeSeq` - save your stuff
4. `close` - and we're done

Now read the bits below to understand what's missing.

### Issues with the Example

Missing Sections: This is a low level API, so headers and sections need to be stored and loaded by the user, see `storeHeader` & `loadHeader` and `storeSection` & `loadSection`, respectively.

No Error Handling: The API is centered around IO and prone to error, each operation checks or sets the `RodFile.err` field. A user of this API needs to handle these appropriately.

# API Notes

## Valid inputs for Rod files

ASTs, hopes, dreams, and anything as long as it and any children it may have support `copyMem`. This means anything that is not a pointer and that does not contain a pointer. At a glance these are:

- string
- objects & tuples (fields are recursed)
- sequences AKA `seq[T]`

## Note on error handling style

A flag based approach is used where operations no-op in case of a preexisting error and set the flag if they encounter one.

## Misc

- 'Prim' is short for 'primitive', as in a non-sequence type

# Types

    RodFile = object
      f*: File
      currentSection*: RodSection
      err*: RodFileError

  Source Edit

    RodFileError = enum
      ok, tooBig, cannotOpen, ioFailure, wrongHeader, wrongSection, configMismatch,
      includeFileChanged

  Source Edit

    RodSection = enum
      versionSection, configSection, stringsSection, checkSumsSection, depsSection,
      numbersSection, exportsSection, hiddenSection, reexportsSection,
      compilerProcsSection, trmacrosSection, convertersSection, methodsSection,
      pureEnumsSection, toReplaySection, topLevelSection, bodiesSection,
      symsSection, typesSection, typeInstCacheSection, procInstCacheSection,
      attachedOpsSection, methodsPerTypeSection, enumToStringProcsSection,
      typeInfoSection, backendFlagsSection, aliveSymsSection

  Source Edit

# Procs

    proc close(f: var RodFile) {....raises: [], tags: [], forbids: [].}

  Source Edit

    proc create(filename: string): RodFile {....raises: [], tags: [], forbids: [].}

  create the file and open it for writing Source Edit

    proc loadHeader(f: var RodFile) {....raises: [IOError], tags: [ReadIOEffect],
                                      forbids: [].}

  Loads the header which is described by `cookie`. Source Edit

    proc loadPrim(f: var RodFile; s: var string) {....raises: [IOError],
        tags: [ReadIOEffect], forbids: [].}

  Read a string, the length was stored as a prefix Source Edit

    proc loadPrim[T](f: var RodFile; x: var T)

  Load a non-sequence/string `T`. Source Edit

    proc loadSection(f: var RodFile; expected: RodSection) {....raises: [IOError],
        tags: [ReadIOEffect], forbids: [].}

  read the bytes value of s, sets and error if the section is incorrect. Source Edit

    proc loadSeq[T](f: var RodFile; s: var seq[T])

  `T` must be compatible with `copyMem`, see `loadPrim` Source Edit

    proc open(filename: string): RodFile {....raises: [], tags: [], forbids: [].}

  open the file for reading Source Edit

    proc storeHeader(f: var RodFile) {....raises: [IOError], tags: [WriteIOEffect],
                                       forbids: [].}

  stores the header which is described by `cookie`. Source Edit

    proc storePrim(f: var RodFile; s: string) {....raises: [IOError],
        tags: [WriteIOEffect], forbids: [].}

  Stores a string. The len is prefixed to allow for later retreival. Source Edit

    proc storePrim[T](f: var RodFile; x: T)

  Stores a non-sequence/string `T`. If `T` doesn't support `copyMem` and is an object or tuple then the fields are written -- the user from context will need to know which `T` to load. Source Edit

    proc storeSection(f: var RodFile; s: RodSection) {....raises: [IOError],
        tags: [WriteIOEffect], forbids: [].}

  update `currentSection` and writes the bytes value of s. Source Edit

    proc storeSeq[T](f: var RodFile; s: seq[T])

  Stores a sequence of `T`s, with the len as a prefix for later retrieval. Source Edit