Type System

The type system is how we define the shape of the data passed between nodes. The node compute function gets access to this data to generate its output. Before diving into the details here is a quick summary of the terminology you need to know.

• Node Is one operation in the processing pipeline.
• Attribute is the common term for everything coming in and out of a Node. There are 3 type of attributes, input, output and configuration.
• Slot is a container for a value, or the lack thereof. It keeps track of changes and can enforce constraints on the value.
• Model is the data structure of a value. What fields does it have. It's also in charge of the serialization of the value.

A Node can have many Attributes, usually at least one input and one output. Each Attribute has one Slot, which tells us what type of data the attribute represents. A Slot has one model, the expected model type and a nullable instance.

To describe the type system in more detail the main concepts are Slot and Model.

Slots

A Slot is a container for a value, which is a nullable instance of a Model. The Slot knows the expected type of Model and can also enforce constraints on the value.

Here is basic example of how Slots and Models are used in code:

node.add_input("A", Slot(value_type=Number))
node.add_input("B", Slot(value_type=Number))
node.add_input("Name", Slot(value_type=String))

While using the base Slot with a given value type work, it is recommended to use one of the many Slot sub-classes that covers your use-case:

node.add_input("Image", ImageSlot(channels=1))
node.add_input("N", NumberInRange(min=0, max=255, default=128))
node.add_input("K", OddIntSlot(default=3))

Lenscraft uses information from the slot to generate the node UI. A more specific slot generally gives you better user experience.

For more about how to implement custom Nodes look at the plugin section.

NumberInRange is an example of a Slot that has a constraint. The constraint can enable runtime validation and it is also used inform the UI. The type NumberInRange tells the attribute UI to render a slider and the constraint values are used to initialize it.

Linking attributes

When we create a link in the UI by dragging an output of one Node to the input of another, we verify if the link can be made using the Slots. Before creating the link we check

if slot1.is_compatible(slot2):
   ...

is_compatible returns True if the value of slot1 can be put into slot2. We are not checking the current value, but the expected value type. In the basic case we check issubclass(slot1.type, slot2.type).

Models

Models describe a data structure that can be used by nodes. In he simplest case a model is just a dumb wrapper around a base value like a int or a str. In more complicated cases it could be a large data structure, a machine learning model, whatever.

The Model defines the data and also how that data can be serialized to disk. When a project is saved node values are saved using the to_dict method on the model. When a project is loaded, the state is parsed by the from_dict method.

Here is a basic example of a Model

class Circle(ValueModel):
    def __init__(self, x, y, r):
        super().__init__()
        self.radius = r
        self.x = x
        self.y = y

    def __eq__(self, other):
        if not isinstance(other, Circle):
            return False

        if self.radius == other.radius and self.x == other.x and self.y == other.y:
            return True

        return False

    def to_dict(self) -> dict:
        return {"x": self.x, "y": self.y, "radius": self.radius}

    @classmethod
    def from_dict(cls, data: dict) -> "Circle":
        return cls(data["x"], data["y"], data["radius"])

The CircleDetection Node uses this model in its output attribute.

Accessing Values

Inside Node.compute you will want to access the input and config values of the node to compute a result. This is done with get_input_value and get_config_value.

def compute(self):
    image = self.get_input_value("Image")
    value = self.get_config_value("Value")

    ...

    self.set_output_value("Result", result)

What does get_input_value actually return?

Unfortunately type hints will not help you here. Strictly speaking it returns any. From the context of the setup method you can see what to expect. The slot defines what model it expects, e.g. ImageSlot expects ImageArray, IntSlot expects a Number.

So, self.get_input_value("Image") returns an instance of ImageArray?

Well... kinda, almost. In most cases get_input_value will try to return the raw data type rather than the model that wraps it. This is because it gets awkward to always need to unpack model classes if you are just working with simple data types.

self.get_input_value("Image") will actually return a numpy.ndarray. In 99% of cases, thats actually want you want to work with. Same applies to base types like int, str, bool, etc. If you are working with a custom model like Circle, you will most likely get the model instance.

self.add_input("Circle", Slot(value_type=Circle))
...
circle = self.get_input_value("Circle")
print(circle.radius)

How is this behavior determined?

If the model has a property value, that value will be returned instead of the model class. Custom Slots can also override this behavior by implementing get_raw_value, which decides what to return.

If you want to skip this behavior and get the model instead, you can use prefer_model=True in get_input_value. This can be helpful if you want to access helpful model methods.

image = self.get_input_value("Image", prefer_model=True)
assert isinstance(image, ImageArray)
min_val, max_val = image.guess_value_range()