RLiterate 2

RLiterate is a graphical tool for doing literate programming. Literate programming is a style of programming in which you don't write source code directly. Instead you write a document which has code snippets interspersed with regular text. It shifts the mindset of the programmer from writing code to writing a document in which the program is presented in small pieces.

This book describes RLiterate, its history, and also includes the complete implementation of the second version.

Showcase

TODO: Show an example how literate programming is done in RLiterate.

Concepts

TODO: Explain concepts and features in greater detail.

Why literate programming?

To me, the central idea in literate programming is that you should write programs for other humans. Only secondary for the machine. I find code easier to understand if I can understand it in isolated pieces. One example where it is difficult to isolate a piece without literate programming is test code. Usually the test code is located in one file, and the implementation in another. To fully understand the piece it is useful to both read the tests and the implementation. To do this you have to find this related information in two unrelated files. I wrote about this problem in 2013 in Related things are not kept together. Literate programming allows you to present the test and the implementation in the same place, yet have the different pieces written to different files. The compiler might require that they are in separate files, but with literate programming, you care first about the other human that will read our code, and only second about the compiler.

Another argument for literate programming is to express the "why". Why is this code here? Timothy Daly talks about it in his talk Literate Programming in the Large. He also argues that programmers must change the mindset from wring a program to writing a book. Some more can be read in Example of Literate Programming in HTML.

Some more resources about literate programming:

Implementation

This chapter contains the complete implementation of the second version of RLiterate.

Files

RLiterate is written in a single Python file that looks like this:

  1. rliterate2.py
#!/usr/bin/env python3

from collections import namedtuple, defaultdict, OrderedDict
from operator import add, sub, mul, floordiv
import base64
import contextlib
import cProfile
import io
import json
import math
import os
import pstats
import sys
import textwrap
import time
import uuid

from pygments.token import Token as TokenType
from pygments.token import string_to_tokentype
import pygments.lexers
import pygments.token
import wx

<<globals>>

<<decorators>>

<<functions>>

<<base base classes>>

<<base classes>>

<<classes>>

if __name__ == "__main__":
    main()

There is also a corresponding test file:

  1. test_rliterate2.py
from unittest.mock import Mock, call

import rliterate2

The remainder of this chapter will fill in the sections in those files to create the complete implementation.

Architecture overview

TODO: Introduce GUI framework, document model, and main frame.

Pattern for passing props

The architecture of the GUI language is to have a top level widget and a function that generates props for it like this:

frame MainFrame %layout_rows {
    ...
}
def main_frame_props(...):
    return {
        ...
    }

Let's say that the main frame has two sub components. It would probably look like this:

frame MainFrame %layout_rows {
    Foo {
        #foo
        %align[EXPAND]
    }
    Bar {
        #bar
        %align[EXPAND]
    }
}
def main_frame_props(...):
    return {
        "foo": foo_props(...),
        "bar": bar_props(...),
    }

def foo_props(...):
    return {
        ...
    }

def bar_props(...):
    return {
        ...
    }

Now let's say that there is a common prop in foo and bar called baz. Should foo_props and bar_props change to the following?

def foo_props(...):
    return {
        "baz": ...,
        ...
    }

def bar_props(...):
    return {
        "baz": ...,
        ...
    }

Or should the common prop be promoted to a main frame prop and passed in the GUI code like this?

frame MainFrame %layout_rows {
    Foo {
        #foo
        baz = #baz
        %align[EXPAND]
    }
    Bar {
        #bar
        baz = #baz
        %align[EXPAND]
    }
}
def main_frame_props(...):
    return {
        "foo": foo_props(...),
        "bar": bar_props(...),
        "baz": ...,
    }

def foo_props(...):
    return {
        ...
    }

def bar_props(...):
    return {
        ...
    }

Both ways are supported by the GUI language.

From a performance perspective, it probably doesn't matter too much. If the common prop is used deep in the widget hierarchy, it needs to be passed at each level if it's passed in the GUI code. If it's inserted in the prop function it can be inserted just where it needs to be.

The pattern with one prop function per widget is quite straight forward. If that is the way to go, perhaps the GUI language can be restricted to only allow that. Not sure if that is a good idea. But I like this pattern, and I will try to follow it.

The problem I encountered with this pattern was that when the column width changed, props for code had to be generated again and code had to be highlighted again. That shouldn't be necessary. But because caching only happened at the paragraph level, and the column width is part of the arguments to paragraph prop, it resulted in a cache miss. The solution is probably to cache deeper down as well.

Main function

  1. rliterate2.py
  2. functions
def main():
    args = parse_args()
    start_app(
        MainFrame,
        create_props(
            main_frame_props,
            Document(args["path"])
        )
    )
  1. rliterate2.py
  2. functions
def parse_args():
    args = {
        "path": None,
    }
    script = sys.argv[0]
    rest = sys.argv[1:]
    if len(rest) != 1:
        usage(script)
    args["path"] = rest[0]
    return args
  1. rliterate2.py
  2. functions
def usage(script):
    sys.exit(f"usage: {script} <path>")

For reference: start_app, create_props.

Main frame

  1. rliterate2.py
  2. classes
frame MainFrame %layout_rows {
    Toolbar {
        #toolbar
        %align[EXPAND]
    }
    ToolbarDivider {
        #toolbar_divider
        %align[EXPAND]
    }
    MainArea {
        #main_area
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions
def main_frame_props(document):
    return {
        "title": format_title(
            document.get(["path"])
        ),
        "toolbar": toolbar_props(
            document
        ),
        "toolbar_divider": toolbar_divider_props(
            document
        ),
        "main_area": main_area_props(
            document
        ),
    }
  1. rliterate2.py
  2. functions
@cache()
def format_title(path):
    return "{} ({}) - RLiterate 2".format(
        os.path.basename(path),
        os.path.abspath(os.path.dirname(path))
    )

Toolbar

  1. rliterate2.py
  2. classes
panel Toolbar %layout_columns {
    %space[#margin]
    ToolbarButton {
        icon    = "quit"
        %margin[#margin,TOP|BOTTOM]
    }
    ToolbarButton {
        icon    = "settings"
        @button = #actions.rotate_theme()
        %margin[#margin,TOP|BOTTOM]
    }
    %space[#margin]
    if (#text_fragment_selection) {
        Panel {
            #bar
            %align[EXPAND]
            %margin[#margin,TOP|BOTTOM]
        }
        %space[#margin]
        ToolbarButton {
            icon    = "bold"
            @button = self._add_text()
            %margin[#margin,TOP|BOTTOM]
        }
        %space[#margin]
    }
}

<<Toolbar>>
  1. rliterate2.py
  2. classes
  3. Toolbar
def _add_text(self):
    selection = self.prop(["selection"])
    self.prop(["actions", "modify_paragraph"])(
        selection.value[-1]["paragraph_id"],
        selection.value[-1]["path"],
        lambda fragments: [{"type": "text", "text": "Hej!"}]+fragments,
        selection._replace(stamp=genid(), value=selection.value[:-1]+[dict(selection.value[-1], **{
            "start": [0, 0],
            "end": [0, 3],
            "cursor_at_start": False,
        })])
    )
  1. rliterate2.py
  2. functions
def toolbar_props(document):
    toolbar_theme = document.get(["theme", "toolbar"])
    selection = document.get(["selection"])
    if (selection.value and
        selection.value[-1].get("what", None) == "text_fragments"):
        text_fragment_selection = True
    else:
        text_fragment_selection = False
    return {
        "background": toolbar_theme["background"],
        "margin": toolbar_theme["margin"],
        "bar": {
            "background": document.get(["theme", "toolbar_divider", "color"]),
            "size": (1, -1),
        },
        "text_fragment_selection": text_fragment_selection,
        "selection": selection,
        "actions": document.actions,
    }

Toolbar divider

  1. rliterate2.py
  2. classes
panel ToolbarDivider %layout_rows {
}
  1. rliterate2.py
  2. functions
def toolbar_divider_props(document):
    toolbar_divider_theme = document.get(["theme", "toolbar_divider"])
    return {
        "background": toolbar_divider_theme["color"],
        "min_size": (
            -1,
            toolbar_divider_theme["thickness"]
        ),
    }

Main area

  1. rliterate2.py
  2. classes
panel MainArea %layout_columns {
    TableOfContents[toc] {
        #toc
        %align[EXPAND]
    }
    TableOfContentsDivider {
        #toc_divider
        @drag = self._on_toc_divider_drag(event)
        %align[EXPAND]
    }
    Workspace {
        #workspace
        %align[EXPAND]
        %proportion[1]
    }
}

<<MainArea>>
  1. rliterate2.py
  2. classes
  3. MainArea
def _on_toc_divider_drag(self, event):
    if event.initial:
        toc = self.get_widget("toc")
        self._start_width = toc.get_width()
    else:
        self.prop(["actions", "set_toc_width"])(
            self._start_width + event.dx
        )
  1. rliterate2.py
  2. functions
def main_area_props(document):
    return {
        "actions": document.actions,
        "toc": toc_props(
            document
        ),
        "toc_divider": toc_divider_props(
            document
        ),
        "workspace": workspace_props(
            document
        ),
    }

Table of contents

  1. rliterate2.py
  2. classes
panel TableOfContents %layout_rows {
    if (#has_valid_hoisted_page) {
        Button {
            label   = "unhoist"
            @button = #actions.set_hoisted_page(None)
            %margin[#margin,ALL]
            %align[EXPAND]
        }
    }
    TableOfContentsScrollArea {
        #scroll_area
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions
def toc_props(document):
    return {
        "background": document.get(
            ["theme", "toc", "background"]
        ),
        "min_size": (
            max(50, document.get(["toc", "width"])),
            -1
        ),
        "has_valid_hoisted_page": is_valid_hoisted_page(
            document,
            document.get(["toc", "hoisted_page"]),
        ),
        "margin": 1 + document.get(
            ["theme", "toc", "row_margin"]
        ),
        "actions": document.actions,
        "scroll_area": toc_scroll_area_props(
            document
        ),
    }
  1. rliterate2.py
  2. functions
def is_valid_hoisted_page(document, page_id):
    try:
        page = document.get_page(page_id)
        root_page = document.get_page()
        if page["id"] != root_page["id"]:
            return True
    except PageNotFound:
        pass
    return False
Scroll area
  1. rliterate2.py
  2. classes
scroll TableOfContentsScrollArea %layout_rows {
    drop_target = "move_page"
    loop (#rows cache_limit=#rows_cache_limit) {
        TableOfContentsRow[rows] {
            $
            __reuse = $id
            __cache = True
            %align[EXPAND]
        }
    }
}

<<TableOfContentsScrollArea>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsScrollArea
_last_drop_row = None

def on_drag_drop_over(self, x, y):
    self._hide()
    drop_point = self._get_drop_point(x, y)
    if drop_point is not None:
        self._last_drop_row = self._get_drop_row(drop_point)
    if self._last_drop_row is not None:
        valid = self.prop(["actions", "can_move_page"])(
            self.prop(["dragged_page"]),
            drop_point.target_page,
            drop_point.target_index
        )
        self._last_drop_row.show_drop_line(
            self._calculate_indent(drop_point.level),
            valid=valid
        )
        return valid
    return False

def on_drag_drop_leave(self):
    self._hide()

def on_drag_drop_data(self, x, y, page_info):
    drop_point = self._get_drop_point(x, y)
    if drop_point is not None:
        self.prop(["actions", "move_page"])(
            self.prop(["dragged_page"]),
            drop_point.target_page,
            drop_point.target_index
        )

def _hide(self):
    if self._last_drop_row is not None:
        self._last_drop_row.hide_drop_line()

def _get_drop_point(self, x, y):
    lines = defaultdict(list)
    for drop_point in self.prop(["drop_points"]):
        lines[
            self._y_distance_to(
                self._get_drop_row(drop_point),
                y
            )
        ].append(drop_point)
    if lines:
        columns = {}
        for drop_point in lines[min(lines.keys())]:
            columns[self._x_distance_to(drop_point, x)] = drop_point
        return columns[min(columns.keys())]

def _get_drop_row(self, drop_point):
    return self.get_widget("rows", drop_point.row_index)

def _y_distance_to(self, row, y):
    span_y_center = row.get_y() + row.get_drop_line_y_offset()
    return int(abs(span_y_center - y))

def _x_distance_to(self, drop_point, x):
    return int(abs(self._calculate_indent(drop_point.level + 1.5) - x))

def _calculate_indent(self, level):
    return (
        (2 * self.prop(["row_margin"])) +
        (level + 1) * self.prop(["indent_size"])
    )
  1. rliterate2.py
  2. functions
def toc_scroll_area_props(document):
    return dict(generate_rows_and_drop_points(document), **{
        "rows_cache_limit": document.count_pages() - 1,
        "row_margin": document.get(
            ["theme", "toc", "row_margin"]
        ),
        "indent_size": document.get(
            ["theme", "toc", "indent_size"]
        ),
        "dragged_page": document.get(
            ["toc", "dragged_page"]
        ),
        "actions": document.actions,
    })
  1. rliterate2.py
  2. functions
def generate_rows_and_drop_points(document):
    try:
        root_page = document.get_page(
            document.get(["toc", "hoisted_page"])
        )
    except PageNotFound:
        root_page = document.get_page(None)
    return generate_rows_and_drop_points_page(
        root_page,
        toc_row_common_props(document),
        document.get(["toc", "collapsed"]),
        document.get(["toc", "dragged_page"]),
        document.get_open_pages(),
        0,
        False,
        0
    )
  1. rliterate2.py
  2. functions
@cache(limit=1000, key_path=[0, "id"])
def generate_rows_and_drop_points_page(
    page,
    row_common_props,
    collapsed,
    dragged_page,
    open_pages,
    level,
    is_dragged,
    row_offset
):
    rows = []
    drop_points = []
    is_collapsed = page["id"] in collapsed
    is_dragged = is_dragged or page["id"] == dragged_page
    rows.append(toc_row_props(
        row_common_props,
        page,
        level,
        is_dragged,
        is_collapsed,
        open_pages
    ))
    if is_collapsed:
        target_index = len(page["children"])
    else:
        target_index = 0
    drop_points.append(TableOfContentsDropPoint(
        row_index=row_offset+len(rows)-1,
        target_index=target_index,
        target_page=page["id"],
        level=level+1
    ))
    if not is_collapsed:
        for target_index, child in enumerate(page["children"]):
            sub_result = generate_rows_and_drop_points_page(
                child,
                row_common_props,
                collapsed,
                dragged_page,
                open_pages,
                level+1,
                is_dragged,
                row_offset+len(rows)
            )
            rows.extend(sub_result["rows"])
            drop_points.extend(sub_result["drop_points"])
            drop_points.append(TableOfContentsDropPoint(
                row_index=row_offset+len(rows)-1,
                target_index=target_index+1,
                target_page=page["id"],
                level=level+1
            ))
    return {
        "rows": rows,
        "drop_points": drop_points,
    }
  1. rliterate2.py
  2. classes
TableOfContentsDropPoint = namedtuple("TableOfContentsDropPoint", [
    "row_index",
    "target_index",
    "target_page",
    "level",
])
Row
  1. rliterate2.py
  2. classes
panel TableOfContentsRow %layout_rows {
    TableOfContentsTitle[title] {
        #title
        @click       = #actions.open_page(#id)
        @drag        = self._on_drag(event)
        @right_click = self._on_right_click(event)
        @hover       = self._set_background(event.mouse_inside)
        %align[EXPAND]
    }
    TableOfContentsDropLine[drop_line] {
        #drop_line
        %align[EXPAND]
    }
}

<<TableOfContentsRow>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def _on_drag(self, event):
    if not event.initial:
        self.prop(["actions", "set_dragged_page"])(
            self.prop(["id"])
        )
        try:
            event.initiate_drag_drop("move_page", {})
        finally:
            self.prop(["actions", "set_dragged_page"])(
                None
            )
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def _on_right_click(self, event):
    event.show_context_menu([
        ("Hoist", self.prop(["level"]) > 0, lambda:
            self.prop(["actions", "set_hoisted_page"])(
                self.prop(["id"])
            )
        ),
    ])
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def _set_background(self, hover):
    if hover:
        self.get_widget("title").update_props({
            "background": self.prop(["hover_background"]),
        })
    else:
        self.get_widget("title").update_props({
            "background": None,
        })
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def get_drop_line_y_offset(self):
    drop_line = self.get_widget("drop_line")
    return drop_line.get_y() + drop_line.get_height() / 2
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def show_drop_line(self, indent, valid):
    self.get_widget("drop_line").update_props({
        "active": True,
        "valid": valid,
        "indent": indent
    })
  1. rliterate2.py
  2. classes
  3. TableOfContentsRow
def hide_drop_line(self):
    self.get_widget("drop_line").update_props({
        "active": False,
    })
  1. rliterate2.py
  2. functions
def toc_row_common_props(document):
    return {
        "row_margin": document.get(
            ["theme", "toc", "row_margin"]
        ),
        "indent_size": document.get(
            ["theme", "toc", "indent_size"]
        ),
        "foreground": document.get(
            ["theme", "toc", "foreground"]
        ),
        "hover_background": document.get(
            ["theme", "toc", "hover_background"]
        ),
        "divider_thickness": document.get(
            ["theme", "toc", "divider_thickness"]
        ),
        "dragdrop_color": document.get(
            ["theme", "dragdrop_color"]
        ),
        "dragdrop_invalid_color": document.get(
            ["theme", "dragdrop_invalid_color"]
        ),
        "placeholder_color": document.get(
            ["theme", "toc", "placeholder_color"]
        ),
        "font": document.get(
            ["theme", "toc", "font"]
        ),
        "actions": document.actions,
    }
  1. rliterate2.py
  2. functions
def toc_row_props(
    row_common_props,
    page,
    level,
    is_dragged,
    is_collapsed,
    open_pages
):
    return {
        "id": page["id"],
        "hover_background": row_common_props["hover_background"],
        "actions": row_common_props["actions"],
        "level": level,
        "title": toc_row_title_props(
            row_common_props,
            page,
            level,
            is_dragged,
            is_collapsed,
            open_pages
        ),
        "drop_line": toc_row_drop_line_props(
            row_common_props
        ),
    }
Title
  1. rliterate2.py
  2. classes
panel TableOfContentsTitle %layout_columns {
    %space[add(#row_margin mul(#level #indent_size))]
    if (#has_children) {
        ExpandCollapse {
            cursor       = "hand"
            size         = #indent_size
            collapsed    = #collapsed
            @click       = #actions.toggle_collapsed(#id)
            @drag        = None
            @right_click = None
            %align[EXPAND]
        }
    } else {
        %space[#indent_size]
    }
    Text {
        #text_props
        %align[EXPAND]
        %margin[#row_margin,ALL]
    }
}
  1. rliterate2.py
  2. functions
def toc_row_title_props(
    row_common_props,
    page,
    level,
    is_dragged,
    is_collapsed,
    open_pages
):
    if page["title"]:
        text = page["title"]
        if is_dragged:
            color = row_common_props["dragdrop_invalid_color"]
        else:
            color = row_common_props["foreground"]
    else:
        text = "Enter title..."
        color = row_common_props["placeholder_color"]
    return dict(row_common_props, **{
        "id": page["id"],
        "text_props": TextPropsBuilder(**dict(row_common_props["font"],
            bold=page["id"] in open_pages,
            color=color
        )).text(text).get(),
        "level": level,
        "has_children": bool(page["children"]),
        "collapsed": is_collapsed,
        "dragged": is_dragged,
    })
Drop line
  1. rliterate2.py
  2. classes
panel TableOfContentsDropLine %layout_columns {
    %space[#indent]
    Panel {
        background = self._get_color(#active #valid)
        %align[EXPAND]
        %proportion[1]
    }
}

<<TableOfContentsDropLine>>
  1. rliterate2.py
  2. classes
  3. TableOfContentsDropLine
def _get_color(self, active, valid):
    if active:
        if valid:
            return self.prop(["color"])
        else:
            return self.prop(["invalid_color"])
    else:
        return None
  1. rliterate2.py
  2. functions
def toc_row_drop_line_props(row_common_props):
    return {
        "indent": 0,
        "active": False,
        "valid": True,
        "min_size": (-1, row_common_props["divider_thickness"]),
        "color": row_common_props["dragdrop_color"],
        "invalid_color": row_common_props["dragdrop_invalid_color"],
    }

Table of contents divider

  1. rliterate2.py
  2. classes
panel TableOfContentsDivider %layout_columns {
}
  1. rliterate2.py
  2. functions
def toc_divider_props(document):
    toc_divider_theme = document.get(["theme", "toc_divider"])
    return {
        "background": toc_divider_theme["color"],
        "min_size": (
            toc_divider_theme["thickness"],
            -1
        ),
        "cursor": "size_horizontal",
    }

Workspace

  1. rliterate2.py
  2. classes
hscroll Workspace %layout_columns {
    %space[#margin]
    loop (#columns) {
        Column {
            $
            %align[EXPAND]
        }
        Panel {
            #divider_panel
            @drag = self._on_divider_drag(event)
            %align[EXPAND]
        }
    }
}

<<Workspace>>
  1. rliterate2.py
  2. classes
  3. Workspace
def _on_divider_drag(self, event):
    if event.initial:
        self._initial_width = self.prop(["page_body_width"])
    else:
        self.prop(["actions", "set_page_body_width"])(
            max(50, self._initial_width + event.dx)
        )
  1. rliterate2.py
  2. functions
def workspace_props(document):
    return {
        "background": document.get(
            ["theme", "workspace", "background"]
        ),
        "margin": document.get(
            ["theme", "workspace", "margin"]
        ),
        "page_body_width": document.get(
            ["workspace", "page_body_width"]
        ),
        "actions": document.actions,
        "columns": columns_props(
            document
        ),
        "divider_panel": {
            "cursor": "size_horizontal",
            "min_size": (document.get(["theme", "workspace", "margin"]), -1)
        }
    }
  1. rliterate2.py
  2. functions
@profile_sub("columns_props")
def columns_props(document):
    return [
        column_props(
            document,
            column,
            document.get(["selection"]).add("workspace", "column", index)
        )
        for index, column
        in enumerate(document.get(["workspace", "columns"]))
    ]
Column
  1. rliterate2.py
  2. classes
vscroll Column %layout_rows {
    %space[#margin]
    loop (#column) {
        Page {
            $
            %align[EXPAND]
        }
        %space[#margin]
    }
}
  1. rliterate2.py
  2. functions
def column_props(document, column, selection):
    column_prop = []
    index = 0
    for page_id in column:
        try:
            column_prop.append(page_props(
                document,
                document.get_page(page_id),
                selection.add("page", index, page_id)
            ))
            index += 1
        except PageNotFound:
            pass
    return {
        "min_size": (
            document.get(["workspace", "page_body_width"]) +
            2*document.get(["theme", "page", "margin"]) +
            document.get(["theme", "page", "border", "size"]),
            -1
        ),
        "margin": document.get(
            ["theme", "workspace", "margin"]
        ),
        "column": column_prop,
    }
Page
  1. rliterate2.py
  2. classes
panel Page %layout_rows {
    PageTopRow {
        #
        %align[EXPAND]
        %proportion[1]
    }
    PageBottomBorder {
        #border
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes
panel PageTopRow %layout_columns {
    PageBody {
        #body
        %align[EXPAND]
        %proportion[1]
    }
    PageRightBorder {
        #border
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes
panel PageBody %layout_rows {
    Title {
        #title
        %margin[#margin,ALL]
        %align[EXPAND]
    }
    loop (#paragraphs) {
        $widget {
            $
            %margin[#margin,LEFT|BOTTOM|RIGHT]
            %align[EXPAND]
        }
    }
}
  1. rliterate2.py
  2. classes
panel PageRightBorder %layout_rows {
    %space[#size]
    Panel {
        min_size   = makeTuple(#size -1)
        background = #color
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. classes
panel PageBottomBorder %layout_columns {
    %space[#size]
    Panel {
        min_size   = makeTuple(-1 #size)
        background = #color
        %align[EXPAND]
        %proportion[1]
    }
}
  1. rliterate2.py
  2. functions
def page_props(document, page, selection):
    page_theme = document.get(["theme", "page"])
    return {
        "body": {
            "id": page["id"],
            "title": page_title_props(
                page,
                document.get(["workspace", "page_body_width"]),
                page_theme,
                selection.add("title"),
                document.actions
            ),
            "paragraphs": paragraphs_props(
                document,
                page["paragraphs"],
                selection.add("paragraphs")
            ),
            "background": page_theme["background"],
            "margin": page_theme["margin"],
        },
        "border": page_theme["border"],
    }
Title
  1. rliterate2.py
  2. classes
panel Title %layout_rows {
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
@cache()
def page_title_props(page, page_body_width, page_theme, selection, actions):
    input_handler = TitleInputHandler(page, page_theme, selection, actions)
    return {
        "text_edit_props": {
            "text_props": dict(
                input_handler.text_props,
                break_at_word=True,
                line_height=page_theme["line_height"]
            ),
            "max_width": page_body_width,
            "selection": selection,
            "selection_color": page_theme["selection_border"],
            "input_handler": input_handler,
            "actions": actions,
        },
    }
  1. rliterate2.py
  2. classes
class TitleInputHandler(StringInputHandler):

    def __init__(self, page, page_theme, selection, actions):
        self.page = page
        self.page_theme = page_theme
        self.selection_trail = selection
        self.actions = actions
        StringInputHandler.__init__(
            self,
            self.page["title"],
            self.selection_trail.get()
        )

    def create_selection(self, values):
        return dict(values, what="page_title", page_id=self.page["id"])

    def build(self):
        builder = TextPropsBuilder(
            **self.page_theme["title_font"],
            selection_color=self.page_theme["selection_color"],
            cursor_color=self.page_theme["cursor_color"]
        )
        self.build_with_selection(builder, self.selection)
        return builder

    def build_with_selection(self, builder, selection):
        if self.data:
            if selection is not None:
                builder.selection_start(selection["start"])
                builder.selection_end(selection["end"])
                if self.selection_trail.active:
                    if selection["cursor_at_start"]:
                        builder.cursor(selection["start"], main=True)
                    else:
                        builder.cursor(selection["end"], main=True)
            builder.text(self.data, index_increment=0)
        else:
            if self.selection_trail.active:
                if selection is not None:
                    builder.cursor(main=True)
            builder.text("Enter title...", color=self.page_theme["placeholder_color"], index_constant=0)
        return builder.get()

    def save(self, new_title, new_selection):
        self.actions["edit_page"](
            self.page["id"],
            {
                "title": new_title,
            },
            self.selection_trail.create(new_selection)
        )
Paragraphs
  1. rliterate2.py
  2. functions
def paragraphs_props(document, paragraphs, selection):
    return [
        paragraph_props(
            dict(
                paragraph,
                meta=build_paragraph_meta(document, paragraph)
            ),
            document.get(["theme", "page"]),
            document.get(["workspace", "page_body_width"]),
            selection.add(index, paragraph["id"]),
            document.actions
        )
        for index, paragraph in enumerate(paragraphs)
    ]
  1. rliterate2.py
  2. functions
def build_paragraph_meta(document, paragraph):
    meta = {
        "variables": {},
        "page_titles": {},
    }
    if paragraph["type"] in ["text", "quote", "image"]:
        build_text_fragments_meta(meta, document, paragraph["fragments"])
    if paragraph["type"] == "list":
        build_list_meta(meta, document, paragraph["children"])
    return meta

def build_list_meta(meta, document, children):
    for child in children:
        build_text_fragments_meta(meta, document, child["fragments"])
        build_list_meta(meta, document, child["children"])

def build_text_fragments_meta(meta, document, fragments):
    for fragment in fragments:
        if fragment["type"] == "variable":
            meta["variables"][fragment["id"]] = document.get_variable(fragment["id"])
        elif fragment["type"] == "reference":
            meta["page_titles"][fragment["page_id"]] = document.get_page(fragment["page_id"])["title"]
  1. rliterate2.py
  2. functions
@cache(limit=1000, key_path=[0, "id"])
def paragraph_props(paragraph, page_theme, body_width, selection, actions):
    BUILDERS = {
        "text": text_paragraph_props,
        "quote": quote_paragraph_props,
        "list": list_paragraph_props,
        "code": code_paragraph_props,
        "image": image_paragraph_props,
    }
    return BUILDERS.get(
        paragraph["type"],
        unknown_paragraph_props
    )(paragraph, page_theme, body_width, selection, actions)
Text
  1. rliterate2.py
  2. classes
panel TextParagraph %layout_rows {
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
@profile_sub("text_paragraph_props")
def text_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": TextParagraph,
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width,
        ),
    }
Quote
  1. rliterate2.py
  2. classes
panel QuoteParagraph %layout_columns {
    %space[#indent_size]
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
@profile_sub("quote_paragraph_props")
def quote_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": QuoteParagraph,
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width-page_theme["indent_size"],
        ),
        "indent_size": page_theme["indent_size"],
    }

List
  1. rliterate2.py
  2. classes
panel ListParagraph %layout_rows {
    loop (#rows) {
        ListRow {
            $
            %align[EXPAND]
        }
    }
}
  1. rliterate2.py
  2. classes
panel ListRow %layout_columns {
    %space[mul(#level #indent)]
    Text {
        #bullet_props
    }
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
@profile_sub("list_paragraph_props")
def list_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": ListParagraph,
        "rows": list_item_rows_props(
            paragraph,
            paragraph["children"],
            paragraph["child_type"],
            page_theme,
            body_width,
            actions,
            selection
        ),
    }
  1. rliterate2.py
  2. functions
def list_item_rows_props(paragraph, children, child_type, page_theme, body_width, actions, selection, path=[], level=0):
    rows = []
    for index, child in enumerate(children):
        rows.append(list_item_row_props(
            paragraph,
            child_type,
            index,
            child,
            page_theme,
            body_width,
            actions,
            selection.add(index, "fragments"),
            path+[index],
            level
        ))
        rows.extend(list_item_rows_props(
            paragraph,
            child["children"],
            child["child_type"],
            page_theme,
            body_width,
            actions,
            selection.add(index),
            path+[index, "children"],
            level+1
        ))
    return rows

def list_item_row_props(paragraph, child_type, index, child, page_theme, body_width, actions, selection, path, level):
    return {
        "level": level,
        "indent": page_theme["indent_size"],
        "bullet_props": dict(
            TextPropsBuilder(
                **page_theme["text_font"]
            ).text(_get_bullet_text(child_type, index)).get(),
            max_width=page_theme["indent_size"],
            line_height=page_theme["line_height"]
        ),
        "text_edit_props": text_fragments_to_text_edit_props(
            paragraph,
            ["children"]+path+["fragments"],
            selection,
            page_theme,
            actions,
            max_width=body_width-(level+1)*page_theme["indent_size"],
            line_height=page_theme["line_height"]
        ),
    }

def _get_bullet_text(list_type, index):
    if list_type == "ordered":
        return "{}. ".format(index + 1)
    else:
        return u"\u2022 "
Code
  1. rliterate2.py
  2. classes
panel CodeParagraph %layout_rows {
    CodeParagraphHeader {
        #header
        %align[EXPAND]
    }
    CodeParagraphBody {
        #body
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
@profile_sub("code_paragraph_props")
def code_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": CodeParagraph,
        "header": code_paragraph_header_props(
            paragraph,
            page_theme,
            body_width
        ),
        "body": code_paragraph_body_props(
            paragraph,
            page_theme,
            body_width
        ),
    }
Header
  1. rliterate2.py
  2. classes
panel CodeParagraphHeader %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
        %margin[#margin,ALL]
    }
}
  1. rliterate2.py
  2. functions
def code_paragraph_header_props(paragraph, page_theme, body_width):
    return {
        "background": page_theme["code"]["header_background"],
        "margin": page_theme["code"]["margin"],
        "text_props": dict(code_paragraph_header_path_props(
            paragraph["filepath"],
            paragraph["chunkpath"],
            page_theme["code_font"]
        ), max_width=body_width-2*page_theme["code"]["margin"]),
    }
  1. rliterate2.py
  2. functions
@cache()
def code_paragraph_header_path_props(filepath, chunkpath, font):
    builder = TextPropsBuilder(**font)
    for index, x in enumerate(filepath):
        if index > 0:
            builder.text("/")
        builder.text(x)
    if filepath and chunkpath:
        builder.text(" ")
    for index, x in enumerate(chunkpath):
        if index > 0:
            builder.text("/")
        builder.text(x)
    return builder.get()
Body
  1. rliterate2.py
  2. classes
panel CodeParagraphBody %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
        %margin[#margin,ALL]
    }
}
  1. rliterate2.py
  2. functions
def code_paragraph_body_props(paragraph, page_theme, body_width):
    return {
        "background": page_theme["code"]["body_background"],
        "margin": page_theme["code"]["margin"],
        "text_props": dict(
            code_paragraph_body_text_props(
                paragraph,
                page_theme
            ),
            max_width=body_width-2*page_theme["code"]["margin"]
        ),
    }
  1. rliterate2.py
  2. functions
@cache(limit=100, key_path=[0, "id"])
def code_paragraph_body_text_props(paragraph, page_theme):
    builder = TextPropsBuilder(**page_theme["code_font"])
    for fragment in apply_token_styles(
        code_body_fragments_props(
            paragraph["fragments"],
            code_pygments_lexer(
                paragraph.get("language", ""),
                paragraph["filepath"][-1] if paragraph["filepath"] else "",
            )
        ),
        page_theme["token_styles"]
    ):
        builder.text(**fragment)
    return builder.get()
  1. rliterate2.py
  2. functions
@profile_sub("apply_token_styles")
def apply_token_styles(fragments, token_styles):
    styles = build_style_dict(token_styles)
    def style_fragment(fragment):
        if "token_type" in fragment:
            token_type = fragment["token_type"]
            while token_type not in styles:
                token_type = token_type.parent
            style = styles[token_type]
            new_fragment = dict(fragment)
            new_fragment.update(style)
            return new_fragment
        else:
            return fragment
    return [
        style_fragment(fragment)
        for fragment in fragments
    ]
  1. rliterate2.py
  2. functions
def build_style_dict(theme_style):
    styles = {}
    for name, value in theme_style.items():
        styles[string_to_tokentype(name)] = value
    return styles
  1. rliterate2.py
  2. functions
def code_body_fragments_props(fragments, pygments_lexer):
    code_chunk = CodeChunk()
    for fragment in fragments:
        if fragment["type"] == "code":
            code_chunk.add(fragment["text"])
        elif fragment["type"] == "chunk":
            code_chunk.add(
                "{}<<{}>>\n".format(
                    fragment["prefix"],
                    "/".join(fragment["path"])
                ),
                {"token_type": TokenType.Comment.Preproc}
            )
    return code_chunk.tokenize(pygments_lexer)
  1. rliterate2.py
  2. functions
def code_pygments_lexer(language, filename):
    try:
        if language:
            return pygments.lexers.get_lexer_by_name(
                language,
                stripnl=False
            )
        else:
            return pygments.lexers.get_lexer_for_filename(
                filename,
                stripnl=False
            )
    except:
        return pygments.lexers.TextLexer(stripnl=False)
Image
  1. rliterate2.py
  2. classes
panel ImageParagraph %layout_rows {
    Image {
        #image
        %align[CENTER]
    }
    ImageText {
        #image_text
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. classes
panel ImageText %layout_columns {
    %space[#indent]
    TextEdit {
        #text_edit_props
        %align[EXPAND]
    }
    %space[#indent]
}
  1. rliterate2.py
  2. functions
@profile_sub("image_paragraph_props")
def image_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": ImageParagraph,
        "image": {
            "base64_image": paragraph.get("image_base64", None),
            "width": body_width,
        },
        "image_text": {
            "indent": page_theme["indent_size"],
            "text_edit_props": text_fragments_to_text_edit_props(
                paragraph,
                ["fragments"],
                selection,
                page_theme,
                actions,
                align="center",
                max_width=body_width-2*page_theme["indent_size"],
            ),
        },
    }
Unknown paragraph
  1. rliterate2.py
  2. classes
panel UnknownParagraph %layout_rows {
    Text {
        #text_props
        %align[EXPAND]
    }
}
  1. rliterate2.py
  2. functions
def unknown_paragraph_props(paragraph, page_theme, body_width, selection, actions):
    return {
        "widget": UnknownParagraph,
        "text_props": dict(
            TextPropsBuilder(**page_theme["code_font"]).text(
                "Unknown paragraph type '{}'.".format(paragraph["type"])
            ).get(),
            max_width=body_width
        ),
    }

Reusable widgets

TextEdit

The Text widget has support for rendering text. It has support for styling text as well as showing selections and cursors. This edit widget adds generic editing capabilities for data that has been projected to text.

Implementing a text editor has two parts: first the data has to be projected as text, and then an input handler needs to respond to events and change the data/selection accordingly.

  1. rliterate2.py
  2. classes
panel TextEdit %layout_rows {
    selection_box = self._box(#selection #selection_color)
    Text[text] {
        #text_props
        max_width  = sub(#max_width mul(2 self._margin()))
        immediate  = self._immediate(#selection)
        focus      = self._focus(#selection)
        cursor     = self._get_cursor(#selection)
        @click     = self._on_click(event #selection)
        @left_down = self._on_left_down(event #selection)
        @drag      = self._on_drag(event #selection)
        @key       = self._on_key(event #selection)
        @focus     = self._on_focus(#selection)
        @unfocus   = self._on_unfocus(#selection)
        %align[EXPAND]
        %margin[self._margin(),ALL]
    }
}

<<TextEdit>>
  1. rliterate2.py
  2. classes
  3. TextEdit
def _box(self, selection, selection_color):
    return {
        "width": 1 if selection.get() else 0,
        "color": selection_color,
    }
  1. rliterate2.py
  2. classes
  3. TextEdit
def _immediate(self, selection):
    return selection.get()
  1. rliterate2.py
  2. classes
  3. TextEdit
def _focus(self, selection):
    if selection.value:
        return selection.stamp
  1. rliterate2.py
  2. classes
  3. TextEdit
def _margin(self):
    width = self.prop(["selection_box", "width"])
    if width > 0:
        return width + 2
    else:
        return 0
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_click(self, event, selection):
    if selection.value:
        return
    index = self._get_index(event.x, event.y)
    if index is not None:
        self.prop(["actions", "set_selection"])(

            selection.create(
                self.prop(["input_handler"]).create_selection({
                    "start": index,
                    "end": index,
                    "cursor_at_start": True,
                })
            )
        )
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_left_down(self, event, selection):
    if not selection.value:
        return
    index = self._get_index(event.x, event.y)
    if index is not None:
        self.prop(["actions", "set_selection"])(
            selection.create(
                self.prop(["input_handler"]).create_selection({

                    "start": index,
                    "end": index,
                    "cursor_at_start": True,
                })
            )
        )
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_drag(self, event, selection):
    if not selection.value:
        return
    if event.initial:
        self._initial_index = self._get_index(event.x, event.y)
    else:
        new_index = self._get_index(event.x, event.y)
        if self._initial_index is not None and new_index is not None:
            self.prop(["actions", "set_selection"])(
                selection.create(
                    self.prop(["input_handler"]).create_selection({
                        "start": min(self._initial_index, new_index),
                        "end": max(self._initial_index, new_index),
                        "cursor_at_start": new_index <= self._initial_index,
                    })
                )
            )
  1. rliterate2.py
  2. classes
  3. TextEdit
def _get_index(self, x, y):
    character, right_side = self.get_widget("text").get_closest_character_with_side(
        x,
        y
    )
    if character is not None:
        if right_side:
            return character.get("index_right", None)
        else:
            return character.get("index_left", None)
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_key(self, event, selection):
    if selection.value:
        self.prop(["input_handler"]).handle_key(
            event,
            self.get_widget("text")
        )
  1. rliterate2.py
  2. classes
  3. TextEdit
def _get_cursor(self, selection):
    if selection.value:
        return "beam"
    else:
        return None
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_focus(self, selection):
    self.prop(["actions", "activate_selection"])(selection.trail)
  1. rliterate2.py
  2. classes
  3. TextEdit
def _on_unfocus(self, selection):
    self.prop(["actions", "deactivate_selection"])(selection.trail)
TextPropsBuilder
  1. rliterate2.py
  2. classes
class TextPropsBuilder(object):

    def __init__(self, **styles):
        self._characters = []
        self._cursors = []
        self._selections = []
        self._base_style = dict(styles)
        self._main_cursor_char_index = None

    def get(self):
        return {
            "characters": self._characters,
            "cursors": self._cursors,
            "selections": self._selections,
            "base_style": self._base_style,
        }

    def get_main_cursor_char_index(self):
        return self._main_cursor_char_index

    def text(self, text, **kwargs):
        fragment = {}
        for field in TextStyle._fields:
            if field in kwargs:
                fragment[field] = kwargs[field]
        index_prefix = kwargs.get("index_prefix", None)
        if index_prefix is None:
            create_index = lambda x: x
        else:
            create_index = lambda x: index_prefix + [x]
        index_increment = kwargs.get("index_increment", None)
        index_constant = kwargs.get("index_constant", None)
        for index, character in enumerate(text):
            x = dict(fragment, text=character)
            if index_increment is not None:
                x["index_left"] = create_index(
                    index_increment + index
                )
                x["index_right"] = create_index(
                    index_increment + index + 1
                )
            if index_constant is not None:
                x["index_left"] = x["index_right"] = create_index(
                    index_constant
                )
            self._characters.append(x)
        return self

    def selection_start(self, offset=0, color=None):
        self._selections.append((
            self._index(offset),
            self._index(offset),
            color
        ))

    def selection_end(self, offset=0):
        last_selection = self._selections[-1]
        self._selections[-1] = (
            last_selection[0],
            self._index(offset),
            last_selection[2]
        )

    def cursor(self, offset=0, color=None, main=False):
        index = self._index(offset)
        self._cursors.append((index, color))
        if main:
            self._main_cursor_char_index = index

    def _index(self, offset):
        return len(self._characters) + offset
StringInputHandler
  1. rliterate2.py
  2. base classes
class StringInputHandler(object):

    def __init__(self, data, selection):
        self.data = data
        self.selection = selection
        builder = self.build()
        self.text_props = builder.get()
        self.main_cursor_char_index = builder.get_main_cursor_char_index()

    @property
    def start(self):
        return self.selection["start"]

    @property
    def end(self):
        return self.selection["end"]

    @property
    def cursor_at_start(self):
        return self.selection["cursor_at_start"]

    @property
    def cursor(self):
        if self.cursor_at_start:
            return self.start
        else:
            return self.end

    @cursor.setter
    def cursor(self, position):
        self.selection = self.create_selection({
            "start": position,
            "end": position,
            "cursor_at_start": True,
        })

    @property
    def has_selection(self):
        return self.start != self.end

    def replace(self, text):
        self.data = self.data[:self.start] + text + self.data[self.end:]
        position = self.start + len(text)
        self.selection = self.create_selection({
            "start": position,
            "end": position,
            "cursor_at_start": True,
        })

    def handle_key(self, key_event, text):
        print(key_event)
        if key_event.key == "\x08": # Backspace
            if self.has_selection:
                self.replace("")
            else:
                self.selection = self.create_selection({
                    "start": self._next_cursor(self._cursors_left(text)),
                    "end": self.start,
                    "cursor_at_start": True,
                })
                self.replace("")
        elif key_event.key == "\x00": # Del (and many others)
            if self.has_selection:
                self.replace("")
            else:
                self.selection = self.create_selection({
                    "start": self.start,
                    "end": self._next_cursor(self._cursors_right(text)),
                    "cursor_at_start": False,
                })
                self.replace("")
        elif key_event.key == "\x02": # Ctrl-B
            self.cursor = self._next_cursor(self._cursors_left(text))
        elif key_event.key == "\x06": # Ctrl-F
            self.cursor = self._next_cursor(self._cursors_right(text))
        else:
            self.replace(key_event.key)
        self.save(self.data, self.selection)

    def _next_cursor(self, cursors):
        for cursor in cursors:
            if self._cursor_differs(cursor):
                return cursor
        return self.cursor

    def _cursor_differs(self, cursor):
        if cursor == self.cursor:
            return False
        builder = TextPropsBuilder()
        self.</