Data GClasses

Time-series, graph database, and resource persistence.

Source: kernel/c/root-linux/src/c_tranger.c, c_treedb.c, c_node.c, c_resource2.c

C_TRANGER¶

Time-range database manager — wraps timeranger2 for CRUD operations on time-series topics.

master answers what the tranger IS. It is configured (SDF_RD) and read at create, but read afterwards it answers the tranger’s own flag: a master that lost its lock while stopped (another process took the store) is a replica, and gobj_read_bool_attr(gobj_tranger, "master") says FALSE (new after 7.25.4; 7.25.4 answered the configuration). open-rt and open-list follow it: a replica’s feed is an rt_disk, which the other master’s appends reach through the disk. The flag moves when the tranger REVIVES, which happens at the first write or the first topic open after tranger2_stop(). Between the stop and that moment it still says what it was, so an open-list on a topic that is already open can still get an rt_mem feed. Read master again after the restart has opened its topics.

Commands¶

Who gets the realtime feed. EV_TRANGER_RECORD_ADDED is published to every subscriber: a remote one keeps to its own feed with a __filter__ on its rt_id. The appends a LIVE list pushes (open-list without return_data) carry the list_id as their rt_id, the same way: __filter__: {rt_id: "pepe"} takes the pushes of list pepe. Up to 7.25.4 they carried no rt_id: a filtered subscriber never got them, and an unfiltered one got every append once per live list of the topic. Since 7.25.5 the event is EVF_AUTHZ_SUBSCRIBE, and read carries the __subscribe_event__ alias. So when the yuno sets enable_subscription_authz, a remote subscription to the feed needs read on the tranger’s service, the permission that open-rt asks. Up to 7.25.4 a user without read could not open a feed, but could subscribe to the event with no filter and get every record of the feeds that other users opened. See YUNO_AUTH.md §4.6.

# the feed, and the subscription of the same session to its own records
command-yuno id=1911 service=tranger command=open-rt rt_id=rt1 topic_name=pp key=1

gobj_subscribe_event(gobj_remote_tranger, "EV_TRANGER_RECORD_ADDED", {
    __filter__: {rt_id: "rt1"}
}, gobj);

A handle is its opener’s: the channel it was opened by, and the user. The feeds, iterators and live lists that a remote session opens are that session’s. close-rt, close-iterator, close-list, get-page and get-list-data from ANOTHER session answer -403 “<role^name>: iterator ‘it1’ is not yours: another session opened it”, and log a warning, “Handle of another session, refused” (with the src and the user); the handle is left as it was. A gobj of the yuno itself (a local caller) is not refused. In 7.25.4 any session could close or read any handle by its id, and print-tranger shows the ids to every read user.

What “session” means depends on how the command arrives:

So through the agent another USER is refused, and a relayed command of no user is refused a handle that a user opened; two sessions of the SAME user through the agent are one owner, and each can close what the other opened -- the channel cannot tell them apart. The handles of a relayed command are not reaped when the operator’s session closes (only a direct session is watched, see open-iterator): close them.

# user alice, through the agent
ycommand -c 'command-yuno id=1911 service=tranger command=open-iterator iterator_id=it1 topic_name=pp key=1'
# user bob, through the agent, the same id
ycommand -c 'command-yuno id=1911 service=tranger command=close-iterator iterator_id=it1'
# -> -403: <role^name>: iterator 'it1' is not yours: another session opened it

The same holds, per connection, for two clients connected to the yuno directly: a GUI that opened it1 keeps it from another GUI, even one of the same user.

open-iterator match conditions (all optional, and 0 or empty means unset): from_t/to_t, from_tm/to_tm, from_rowid/to_rowid, backward, and the user_flag conditions (user_flag, not_user_flag, user_flag_mask_set, user_flag_mask_notset). They are ANDed, and every one is honored per record.

Several keys in one iterator. Give rkey instead of key. The iterator lays the matching keys end to end, sorted by key, in the same order tr2list prints a topic. get-page positions are positions in that concatenation. Each record gives its key in __md_tranger__.key, because one page can hold more than one key. The order is not a time order: a rowid counts inside one key only. To see the records by time, sort the page on the client. The match conditions apply to each key separately. key and rkey together is an error.

command-yuno id=<id> service=<tranger> command=open-iterator iterator_id=all1 topic_name=binaries rkey=.*
command-yuno id=<id> service=<tranger> command=get-page iterator_id=all1 from_rowid=1 limit=100
command-yuno id=<id> service=<tranger> command=close-iterator iterator_id=all1

The answer of open-iterator also gives keys, the number of keys it found.

A key deleted under it is a deleted key, born again or not. A multi-key iterator holds no tranger2 iterator between pages, so it keeps one realtime feed over the topic (only_md, fed to nobody) for its key_deleted callback: the next get-page after a delete-key of one of its keys answers -1 “was deleted” and closes the iterator, like a one-key iterator does, even when the key was created again in between. Open it again to page the reborn key. Only in the process that deleted the key (the master): the callback fires in the delete itself, so a REPLICA’s feed never hears it. There the next get-page reads files that are gone: it fails on the read, logs “Key gone from disk while its iterator was open: deleted (by the master?)”, and the iterator stays open. Close it and open it again.

An id is free again the moment its handle is gone. A handle goes with its topic (a close and reopen, delete-topic + create-topic, a stop/start of the service): the entry is dropped at the stop, after the delete-topic, when a get-page answers “closed with its topic”, or when open-iterator / open-rt / open-list meet the id again — so the same id opens a new handle, with data, instead of “already open” and no data.

The keys watch is not a client’s feed. It is named <iterator_id>^__keys__ and opened under a creator of its own, so a client’s open-rt rt_id=all1^__keys__ is another feed: until after 7.25.3 the two collided, open-iterator answered -1, and a get-page of a dead iterator closed the client’s feed. It records only the deletes of the keys its iterator pages.

What it costs. The open counts the rows of each key once (for a filtered iterator that is building and dropping each key’s index) and keeps only a number per key; a get-page opens the keys its page touches, reads, and closes them. Until after 7.24.1 it kept one tranger2 iterator per key for its whole life — 1000 keys with 60 daily files was +147 MB for one card. A key the iterator does not filter is counted LIVE at every get-page, like a one-key iterator: a backward page (newest first) shows the rows appended since the open. A filtered key keeps the count of the rows its filter matched at open (after 7.25.3; before, every key kept the count of the open, so the whole-topic card never showed a new row): a row appended to it after the open is never paged, although each page indexes the key again -- counting it would mean indexing every filtered key on every page. A key CREATED after the open is not in the iterator: open it again to see it.

Positions move under appends. Because an unfiltered key is counted live, a row appended to a key moves the positions after it by one. Paging forward after an append to an EARLIER key (or backward after an append to a LATER key), the next page repeats the last row of the page before; an append never makes a page skip a row. Every record carries its key and rowid, so a client that must not show a row twice drops a repeated (key, rowid). For example, with key I holding 3 rows and key J 2:

command-yuno id=<id> service=<tranger> command=open-iterator iterator_id=ij topic_name=t rkey=^[IJ]$
command-yuno id=<id> service=<tranger> command=get-page iterator_id=ij from_rowid=1 limit=4
#   I#1 I#2 I#3 J#1
#   (a row is appended to I)
command-yuno id=<id> service=<tranger> command=get-page iterator_id=ij from_rowid=5 limit=4
#   J#1 J#2         <- J#1 again

Bound the reading of a large topic anyway: a negative from_rowid reads the last N records of EACH key, and is what gui_treedb’s whole-topic card starts with:

command-yuno id=<id> service=<tranger> command=open-iterator iterator_id=all1 topic_name=binaries rkey=.* from_rowid=-100

A tranger record carries two independent timestamps, and a browser of raw records needs both:

They diverge whenever data is backfilled or a device uploads a buffered batch late. Both are expressed in the topic’s unit — seconds, unless its system_flag sets sf_t_ms / sf_tm_ms (ask topics expanded=1).

What a command is handed is the caller’s data¶

Every answer starts with the yuno that answers (<role^name>: ), a -403 too: -403 “<role^name>: No permission to ‘read’ in service ‘tranger’” (new after 7.26.9; “What topic_name?”, “What list_id?” and the -403s named nobody).

The data a command is handed is checked BEFORE the tranger, which logs an ERROR with a stack for it (a CRITICAL for a topic name longer than NAME_MAX: its mkdir failed, and the default on_critical_error EXITS the yuno). A refusal answers -1 with the cause, and its log depends on who sent the command, as in command_parser():

The authz refusal “Handle of another session, refused” stays a WARNING for both. A name an answer repeats is capped at 64 bytes, never in the middle of a character.

A peer writes a treedb’s topics through the treedb, never through its tranger: C_NODE validates the record, keeps the immutable seeds and asks its own permissions. A raw record appended to users WAS that node at the next load (the newest record of a key wins), a delete-key took a node behind the treedb’s back, and a delete-topic took a topic of a running treedb. A C caller is its own owner, and its writes are taken. The gui_treedb tranger view’s delete key button is refused on a treedb’s own topics for this reason.

command-yuno id=<id> service=tranger_treedb_x command=add-record topic_name=users record='{"id":"raw"}'
# -> -1 <role^name>: 'users' is a topic of a treedb or a msg2db: it is written through it, not by add-record
command-yuno id=<id> service=tranger command=open-iterator iterator_id=it1 topic_name=pp key=../../other/keys/K
# -> -1 <role^name>: key '../../other/keys/K' is not one directory component

A topic of a wrong shape (create-topic). The why of the answer:

command-yuno id=<id> service=tranger command=create-topic topic_name=__x jn_cols='{"id":"","tm":0}'
# -> -1 <role^name>: cannot create topic '__x': a name starting with '__' is of the system
command-yuno id=<id> service=tranger command=create-topic topic_name=pp system_flag='sf_string_key|sf_int_key'
# -> -1 <role^name>: cannot create topic 'pp': more than one key type in system_flag
command-yuno id=<id> service=tranger command=create-topic topic_name=pp jn_var='{"topic_version":3}'
# -> -1 <role^name>: cannot create topic 'pp': a new topic_version of a topic that exists needs its jn_cols (an empty dict for a topic with no columns)
command-yuno id=<id> service=tranger command=create-topic topic_name=events jn_cols='{}' jn_var='{"topic_version":3}'
# -> 0 <role^name>: Topic created: 'events'    (a topic with no columns by design)

A record of a wrong shape (add-record). The why of the answer:

command-yuno id=<id> service=tranger command=add-record topic_name=pp record='{"id":"../x","tm":1}'
# -> -1 <role^name>: record refused by topic 'pp': a pkey with a '/' or a leading '.'
command-yuno id=<id> service=tranger command=add-record topic_name=pp record='{"id":"1","tm":1}' __t__=17592186044416
# -> -1 <role^name>: record refused by topic 'pp': a __t__ larger than the time a record keeps (44 bits)
command-yuno id=<id> service=tranger command=add-record topic_name=pp record='{"id":"1","tm":"@99999999999999 +0000"}'
# -> -1 <role^name>: record refused by topic 'pp': a time in its tkey that a record cannot keep (negative, or past 44 bits)
command-yuno id=<id> service=tranger command=add-record topic_name=pp_ms record='{"id":"1","tm":"@20000000000 +0000"}'
# -> the same refusal: pp_ms is sf_tm_ms, and 2e10 seconds are 2e13 milliseconds, past 44 bits
command-yuno id=<id> service=tranger command=add-record topic_name=pp_ms record='{"id":"1","tm":"@1700000000 +0000"}'
# -> 0 <role^name>: record added to topic 'pp_ms', rowid 1

A NUL reaches a record only in the kw of a frame, which is decoded with JSON_ALLOW_NUL; the json reader of the command line refuses a \u0000. The kw of an add-record of a peer, and its answer:

{"topic_name": "pp", "record": {"id": "K\u0000evil", "tm": 1}}

-1 <role^name>: record refused by topic 'pp': a string with a NUL in it

A replica answers READ-ONLY to the master-only commands: add-record, delete-key, delete-topic, and the create-topic of a NEW topic (no directory of that name). Each one asks at its own place, and the READ-ONLY answer logs nothing (“<role^name>: tranger ‘tranger’ is READ-ONLY, this yuno is not its master: delete-key runs on the master”):

Up to 7.26.9 the tranger refused them with an ERROR (“Only master can delete”, “Not found and no master”).

# on a replica
command-yuno id=<id> service=tranger command=delete-key topic_name=pp key=K force=1
# -> -1 <role^name>: tranger 'tranger' is READ-ONLY, this yuno is not its master: delete-key runs on the master   (no log)
command-yuno id=<id> service=tranger command=delete-key topic_name=nosuch key=K force=1
# -> -1 <role^name>: Topic not found: 'nosuch'   (a WARNING "topic not found, refused")

C_TREEDB¶

Hierarchical tree database manager — manages TreeDB instances on top of timeranger with JSON schema support.

Key attributes¶

Commands¶

Events. EV_OPEN_TREEDB and EV_CLOSE_TREEDB are public events of C_TREEDB, and they are NOT implemented: each one answers -1 and logs an ERROR, “EV_OPEN_TREEDB is not implemented: nothing opened, use the command open-treedb” (and the same for EV_CLOSE_TREEDB), with src and treedb_name. In 7.25.4 the open did nothing and answered 0, and neither said anything. Open and close a treedb with the commands, which ask their permission. From outside, close-treedb closes only a treedb that a peer opened. While the yuno plays, it needs force=1, and only the user whose open-treedb opened it gets that force. While the yuno is paused, any user with open-close closes it, with force or without (see close-treedb above). A treedb that the yuno’s code opened is never closed from outside, playing or paused: its code closes it through the yuno’s own lifecycle (pause-yuno + play-yuno for a yuno that opens its treedbs at play; a restart for mqtt_broker, which opens its treedb at its start).

The case of the name (new after 7.26.9). The service registry finds a service by its name in any case, and a closed treedb is a directory, whose name is case-sensitive. So C_TREEDB reads a name in three ways:

• The name of an OPEN treedb is taken in any case. Every command then acts under the name the treedb was opened with, and the answer names that name: TREEDB_FOO closes treedb_foo, and the answer says 'treedb_foo'. An open-treedb of TREEDB_FOO answers “<role^name>: treedb ‘TREEDB_FOO’ is already open here: close-treedb first, nothing was changed”, with the name as it was written.

• The name of a treedb the yuno’s CODE opened is refused to a peer in any case, also while that treedb is closed: close-treedb, open-treedb and delete-treedb of TREEDB_MQTT_BROKER are refused as for treedb_mqtt_broker. While it is closed, the answer names it as the peer wrote it. A name whose services are that treedb’s (tranger_<it>, or <x> when the code opened tranger_<x>), in any case, does not open either: while the treedb is open, its services exist and the answer is “a service named ... exists already” (see open-treedb above); while it is closed, the peer gets its own refusal (see What a peer hands C_TREEDB).

• Any other name of a closed treedb is taken as it is written: TREEDB_FOO and treedb_foo are two stores, two directories of path. Only one of them can be open at a time, because they share their service names: with TREEDB_FOO open, open-treedb treedb_name=treedb_foo answers “<role^name>: treedb ‘treedb_foo’ is already open here: close-treedb first, nothing was changed”.

For example, on a mqtt_broker <broker> where alice opened treedb_foo and treedb_bar with open-treedb:

# the broker plays; bob, another user, forces the close of alice's treedb_foo: refused
command-yuno id=<broker> service=treedbs command=close-treedb treedb_name=treedb_foo force=1
# -> -1 <role^name>: cannot force the close of treedb 'treedb_foo' while the yuno plays: this user did not open it, and whoever did may hold its handles. Use pause-yuno + play-yuno

# alice forces it, with the name in another case: closed, and the answer names it as it was opened
command-yuno id=<broker> service=treedbs command=close-treedb treedb_name=TREEDB_FOO force=1
# -> 0 <role^name>: treedb closed: 'treedb_foo'

# the broker paused: the treedb its code opened is refused, with force or without
pause-yuno id=<broker>
command-yuno id=<broker> service=treedbs command=close-treedb treedb_name=treedb_mqtt_broker
# -> -1 <role^name>: cannot close treedb 'treedb_mqtt_broker': the code of the yuno opened it and holds its handles, playing or paused; the yuno's own lifecycle closes it (pause-yuno + play-yuno, or a restart of the yuno)
# the same treedb named in another case: refused the same way, and the answer names it as it was opened
command-yuno id=<broker> service=treedbs command=close-treedb treedb_name=TREEDB_MQTT_BROKER force=1
# -> -1 <role^name>: cannot close treedb 'treedb_mqtt_broker': the code of the yuno opened it and holds its handles, playing or paused; the yuno's own lifecycle closes it (pause-yuno + play-yuno, or a restart of the yuno)
# alice's treedb_bar, closed by bob while the broker is paused: no force needed
command-yuno id=<broker> service=treedbs command=close-treedb treedb_name=treedb_bar
# -> 0 <role^name>: treedb closed: 'treedb_bar'
play-yuno id=<broker>

# a db_history yuno: its code closes its treedbs at pause and opens them at play
pause-yuno id=<db_history>
play-yuno id=<db_history>

What a peer hands C_TREEDB¶

A command whose kw carries a __username__ came from OUTSIDE: C_IEVENT_SRV puts it there, and the agent stamps the operator in a command-yuno. What such a command hands is the peer’s data, and a mistake in it is the peer’s: C_TREEDB refuses it with a WARNING (msgset: MSGSET_PARAMETER, no stack) and -1 that says why. A command without one comes from C code of the yuno, and the same refusal is a broken contract: an ERROR with a stack. Every value is read AFTER the permission check, in every command (new after 7.26.9 for open-treedb, close-treedb, delete-treedb, create-topic, delete-topic and diff-schema): a peer without the permission makes the log write nothing of its values.

An open-treedb from outside is checked before anything is made, in this order, and before the C_TRANGER of the treedb is created: a refused open leaves no store directory and no service (up to 7.26.9 a bad name or a missing schema left the directory and the C_TRANGER / C_NODE services behind, a name longer than NAME_MAX was a CRITICAL of the tranger, and the CRITICAL “Cannot create topic of the schema” EXITED the yuno with the exit_on_error the peer passed). The answer is -1, in one of these forms:

What is refused, in the order it is asked:

command-yuno id=<id> service=treedbs command=open-treedb treedb_name=__mine treedb_schema='{"id":"__mine","schema_version":1,"topics":[]}'
# -> -1 <role^name>: cannot open treedb '__mine': a name starting with '__' is of the system, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x filename_mask=%Y/%m treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': filename_mask '%Y/%m' has a path separator, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x exit_on_error=2 treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': exit_on_error is set by the yuno's code, not by a command, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x treedb_schema='{"id":"treedb_x","schema_version":1,"topics":[{"id":"users","topic_name":"people","pkey":"id","system_flag":"sf_string_key","topic_version":1,"cols":{"id":{"header":"Id","type":"string","flag":["persistent"]}}}]}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': topic 'users' of its schema: a topic whose topic_name is not its id: the readers of the schema take one or the other, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x treedb_schema='{...}' initial_load='{"users":[{"name":"x"}]}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': a seed of initial_load without its id, a string, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x treedb_schema='{"id":"treedb_x","schema_version":1,"topics":[{"id":"a","pkey":"id","system_flag":"sf_string_key","topic_version":1,"main_topic":true,"cols":{"id":{"header":"Id","type":"string","flag":["persistent"]}}}]}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': topic 'a' of its schema: a topic marked main_topic that is not hooked to itself, nothing was changed
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_x treedb_schema='{"id":"treedb_x","schema_version":1}'
# -> -1 <role^name>: cannot open treedb 'treedb_x': its schema: a schema without topics (a list or a dict of them), nothing was changed
# treedb_y: the yuno's code opened it in this C_TREEDB, and closed it
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_y treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'treedb_y': it is opened by the code of the yuno, and a peer cannot open it, nor its store, even while it is closed; nothing was changed
# the same, named in another case: the answer names it as it was written
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=TREEDB_Y treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'TREEDB_Y': it is opened by the code of the yuno, and a peer cannot open it, nor its store, even while it is closed; nothing was changed
# a name whose C_NODE would take the service name of treedb_y's tranger
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=tranger_treedb_y treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'tranger_treedb_y': its services would take the names of treedb 'treedb_y', which the code of the yuno opens, and a peer cannot take them even while it is closed; nothing was changed
# the other way: the code opened a treedb named tranger_logs, and closed it; a peer's Logs would take its name with its own tranger
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=Logs treedb_schema='{...}'
# -> -1 <role^name>: cannot open treedb 'Logs': its services would take the names of treedb 'tranger_logs', which the code of the yuno opens, and a peer cannot take them even while it is closed; nothing was changed

The topic of a peer’s schema, written so that it opens:

{
    "id": "treedb_x",
    "schema_version": 1,
    "topics": [
        {
            "id": "users",
            "pkey": "id",
            "system_flag": "sf_string_key",
            "topic_version": 1,
            "pkey2s": "email",
            "cols": {
                "id":    {"header": "Id",    "type": "string", "flag": ["persistent", "required"]},
                "email": {"header": "Email", "type": "string", "flag": ["persistent"]}
            }
        }
    ]
}

A main_topic mark, written so that it opens: ONE topic of the schema, and hooked to itself -- a hook column whose map names its own topic (here children, filled through the parent_id fkey of the child):

{
    "id": "departments",
    "pkey": "id",
    "system_flag": "sf_string_key",
    "topic_version": 1,
    "main_topic": true,
    "cols": {
        "id":        {"header": "Id",       "type": "string", "flag": ["persistent"]},
        "parent_id": {"header": "Parent",   "type": "string", "flag": ["fkey"]},
        "children":  {"header": "Children", "type": "array",  "flag": ["hook"], "hook": {"departments": "parent_id"}}
    }
}

An open-treedb from the yuno’s C code goes as it always went (the agent, the controlcenter, a db_history, the MQTT broker): none of the checks above is asked of it, and the name rule neither. What the treedb refuses of its schema is logged and left out: the treedb logs its cause (an ERROR, for example “Topic refused: bad columns”), then the CRITICAL “Cannot create topic of the schema”, logged with the tranger’s on_critical_error -- the exit_on_error of the open. With LOG_OPT_EXIT_ZERO ("exit_on_error": 2) that CRITICAL EXITS the yuno; with 0, the default of the command, the treedb opens without that topic. An open-treedb that answers -1 is the caller’s to judge: the agent EXITS, and the yunos of the projects go on with no treedb. A topic that is already ON DISK keeps the pkey2s and the list ids it ran with, each fault an ERROR (see treedb_create_topic()). A C caller can open a treedb named __x, and close it.

/*  the open of the yuno's code: no __username__ in the kw  */
json_t *jn_resp = gobj_command(gobj_treedbs, "open-treedb",
    json_pack("{s:s, s:s, s:i, s:o}",
        "treedb_name", "treedb_x",
        "filename_mask", "%Y-%m-%d",
        "exit_on_error", 0,             // a refused topic is left out, the yuno goes on
        "treedb_schema", jn_schema      // owned
    ),
    gobj
);

The seeds of a peer are its data at every start. For an external open, C_TREEDB creates the C_NODE with peer_initial_load set, and what the seeds get wrong (a topic the treedb has not, a parent ref that names no parent, a hook that does not take the seed’s topic) is a WARNING at the open, and again at every later start of that C_NODE (the seeds run in its mt_start: a stop and a start of the service applies them again). A pause-yuno + play-yuno does not start it again: the yunos of today destroy their C_TREEDB at pause, the treedbs a peer opened with it, or (mqtt_broker) keep it running. The seeds of a C caller keep their ERRORs. The store and the schema are opened OUTSIDE that window: a store that does not open is an ERROR whoever opens it.

For example, a peer’s seeds with a topic the treedb has not (nope) and a ref through a hook that groups does not have (nohook), in a schema where items has the fkey group_id:

command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_seeded treedb_schema='{...}' initial_load='{"nope":[{"id":"x"}],"groups":[{"id":"g1"}],"items":[{"id":"i2","group_id":"groups^g1^nohook"}]}'
# -> 0 <role^name>: treedb opened: 'treedb_seeded'
#    WARNING "initial_load: topic not found in treedb"   (topic_name: nope)
#    WARNING "initial_load: bad parent ref"              (why: its hook is not a hook of that topic that links the seed's topic)
# a stop and a start of the service treedb_seeded: the same two WARNINGs again
# the same initial_load in an open-treedb of the yuno's C code: the same two lines as ERRORs, with a stack

create-topic from outside checks a NEW topic as a topic of a peer’s schema is (the treedb’s own checks, a name the store keeps, the store, a topic of another database of the store, the names that must fit, a topic_version past 2147483647), and its file columns against the hooks that the __assets__ of the open treedb already has. A topic the treedb has is answered as it is, before any of these. From the yuno’s C code none of them runs: the command goes as it went before 7.27.0, straight to treedb_create_topic(), which judges the topic and logs what it refuses as an ERROR. A name longer than NAME_MAX is one of them: the ERROR “Topic refused: bad name”, and the tranger is never reached. Only a NEW topic is refused there for its pkey2s and the length of its list ids (see treedb_create_topic()).

command-yuno id=<id> service=treedbs command=create-topic treedb_name=treedb_x topic_name=.blobs cols='{"id":{"header":"Id","type":"string","flag":["persistent"]}}'
# -> -1 <role^name>: cannot create topic '.blobs' in treedb 'treedb_x': a name starting with '.' is a hidden file of the store
command-yuno id=<id> service=treedbs command=create-topic treedb_name=treedb_x topic_name=notes pkey2s=id cols='{"id":{"header":"Id","type":"string","flag":["persistent"]}}'
# -> -1 <role^name>: cannot create topic 'notes' in treedb 'treedb_x': a pkey2 cannot be the pkey 'id'
command-yuno id=<id> service=treedbs command=create-topic treedb_name=treedb_x topic_name=notes topic_version=2147483648 cols='{"id":{"header":"Id","type":"string","flag":["persistent"]}}'
# -> -1 <role^name>: cannot create topic 'notes' in treedb 'treedb_x': a topic_version that is not an integer from 0 to 2147483647

/*  the yuno's code: no __username__, treedb_create_topic() judges it  */
json_t *jn_resp = gobj_command(gobj_treedbs, "create-topic",
    json_pack("{s:s, s:s, s:{s:{s:s, s:s, s:[s]}}}",
        "treedb_name", "treedb_x",
        "topic_name", "notes",
        "cols", "id", "header", "Id", "type", "string", "flag", "persistent"
    ),
    gobj
);
/*  the same with a topic_name of 300 bytes: the ERROR "Topic refused: bad
 *  name" (why: "the topic name breaks the name rule (empty, '.', '..', '/',
 *  '`', or longer than NAME_MAX)"), nothing on disk, and jn_resp is
 *  -1 "<role^name>: cannot create topic '<name>' in treedb 'treedb_x' (see the log)"  */

A topic of another database. The store of a treedb is the tranger of other databases too: the db_history yunos and the MQTT broker open a msg2db msg2db_alarms on the tranger of their treedb, so the store treedb_wattyzer/ holds the topic alarms/ and the file msg2db_alarms.msg2db_schema.json beside the topics of the treedb. Adopted by the treedb, such a topic was the treedb’s: its records were loaded as nodes and written by C_NODE, a higher topic_version rewrote its topic_cols.json and topic_var.json under the msg2db (which never wrote them back: its own version is lower), and a delete-topic took the directory and every record. So a PEER’s new topic is refused when ANOTHER database of the store says, positively, that the topic is its own (new after 7.26.9):

It is asked of a peer’s create-topic, of each topic of a peer’s open-treedb schema, and of each topic of a draft that a peer’s save-schema or apply-schema publishes; a peer’s delete-topic asks the same evidence (see delete-topic above). A topic the treedb holds now is its own. A directory that no other database names is taken as it always was: a topic the treedb had and DROPPED comes back with its records (nothing keeps the name of a dropped topic), a topic a create-topic made and nobody published, a plain topic of a C_TRANGER. A schema file left behind by a msg2db that is gone still names its topics: only a C caller, or removing the file, gets past it. A C caller is asked none of this, as before.

# db_history_wz: the msg2db `msg2db_alarms` is open on the tranger of treedb_wattyzer
command-yuno id=<db_history_wz> service=treedbs command=create-topic treedb_name=treedb_wattyzer topic_name=alarms topic_version=2 cols='{"id":{"header":"Id","type":"string","flag":["persistent"]}}'
# -> -1 <role^name>: cannot create topic 'alarms' in treedb 'treedb_wattyzer': a topic of a msg2db open on the store of the treedb: it is the msg2db's

# a peer's own treedb: opened with the topics items and x, closed (force=1),
# opened with items only, closed; opened again with items and x:
command-yuno id=<id> service=treedbs command=open-treedb treedb_name=treedb_p treedb_schema='{"id":"treedb_p","schema_version":3,"topics":[<items>,<x>]}'
# -> 0 <role^name>: treedb opened: 'treedb_p'      (x is back, with the records it had)

The MQTT broker opens its treedb as C code. It held the yuno’s __username__ in the kw of its own open-treedb and close-treedb, which made them a peer’s (its impose_c_schema was refused). It holds it as an attribute now, as the agent and the controlcenter do, and its permission is asked with it (see mqtt_broker).

C_NODE¶

Node resource interface for TreeDB — full CRUD and graph operations on tree nodes with linking, snapshots, and import/export.

Key attributes¶

Commands¶

Every comment starts with the yuno that answers (<role^name>: ), and a failure says its own cause and points at the log: create-node answers <role^name>: Node created, 'item01' of topic 'items' or <role^name>: cannot create the node of topic 'items' (see the log); shoot-snap, snap-content, nodes, instances, parents, children, jtree, desc, links, hooks, topics and import-assets the same (new after 7.25.4). In 7.25.4 they answered gobj_log_last_message(), the process-global buffer of the last ERROR of anybody -- a stale or unrelated cause -- and most answers carried no yuno. The exception is the refusal of a permission, “No permission to ‘<perm>’ in service ‘<treedb>’”, which names its service. instances answers -1 when it cannot list (it answered 0). The test walks C_NODE’s command table (tests/c/c_node_authz), so a command added with a bare comment fails it. The log says its own cause too: an update of a node that does not exist, with create and a create the library refuses, logs the library’s error and then “Cannot update node: it does not exist and it cannot be created (see the previous log)” (new after 7.25.4; 7.25.4 logged the last message again as its own).

Permissions¶

Every command that reads or writes the treedb asks for a permission, inside the command, with or without the global enable_command_authz gate. It matters only in a yuno with an authz checker (C_AUTHZ).

Only help and authzs ask nothing. system-schema, trace and the set-link-events display answered anyone until after 7.24.1; the test walks C_NODE’s command table (tests/c/c_node_authz), so a command added without a permission fails it.

read also guards the treedb’s feed: the five EV_TREEDB_NODE_* events are EVF_AUTHZ_SUBSCRIBE, and read carries the __subscribe_event__ alias, so in a yuno that sets enable_subscription_authz a peer needs read on the treedb service to subscribe to them (new after 7.25.4; in 7.25.4 any authenticated user got the feed). The gate is off by default, and it goes in the yuno’s config:

{
    "yuno": {
        "enable_subscription_authz": true
    }
}

A peer without read is refused and the channel stays open: ERROR “No permission to subscribe event”, with service, event, permission: read and username. See YUNO_AUTH.md §4.6 and the test tests/c/c_subscription_authz, which subscribes to a real C_NODE.

update-node with options.create=1 also asks for create when the node does not exist yet. A refusal answers -403, and nothing is written.

options.create_only=1 makes a NEW node and refuses one that exists (“Node already exists”); options.create=1 alone is an upsert, and a taken id there is an update of the existing record. A table’s +New sends create_only (gobj-ui 7.23.194). Both ask for create when the node does not exist: create_only alone used to create under update.

command-yuno id=<id> service=<treedb> command=update-node topic_name=users record='{"id":"bob","username":"Bob"}' options='{"create_only":1}'

An update-node with autolink whose record names a link that cannot be made saves the record and answers -1: “<role^name>: node ‘x’ saved, but some of its links were NOT changed: a link it names cannot be made, and that column was left as it was (see the log)”. The fkey column keeps the links it had (see treedb_replace_links()); it used to answer “Node update!”. The answer is the update’s own, even when a subscriber of its events updates the same service before it answers (until 7.25.3 the nested update reset it to a success).

ycommand -c 'command-yuno id=<id> service=<treedb> command=delete-node topic_name=items record={"id":"item1"} options={"ignore_snaps":1}'
# -403 No permission to 'create' in service '<treedb>'   (a role with 'delete' only)

A replica writes nothing, whoever asks. The commands answer “READ-ONLY” before anything is read. From C, every write method refuses before it moves anything: gobj_update_node() returns NULL for every update but a volatil one (memory only) -- the door C_AUTHZ uses for a user created from an event, which no command guards; gobj_link_nodes(), gobj_unlink_nodes() and gobj_delete_node() return -1 and log “Cannot link nodes / unlink nodes / delete a node on a READ-ONLY replica” (new after 7.25.4; 7.25.4 moved the links in memory first and met the refused save last); gobj_create_node() is refused by treedb_create_node(); and shooting or activating a snap by the treedb (“Only master can ...”).

/*  On a replica: -1, nothing moved  */
int ret = gobj_delete_node(gobj_node, "items",
    json_pack("{s:s}", "id", "item00"), json_pack("{s:b}", "force", 1), src);

An autolink write that fails at its save changes nothing it wrote. update-node (or gobj_update_node()) with autolink writes the fields, the links and the save as ONE write (treedb_update_node_and_links()). When the append fails (a full disk, a store gone), it answers NULL / -1. The fields and the links go back in memory to what the disk has, and none of the events of the write is told (EV_TREEDB_NODE_LINKED, EV_TREEDB_NODE_UPDATED):

• create + autolink: the create is on disk, with its ordinary fkeys EMPTY (a create stores none: only its file columns, which it links itself, and, for a secondary instance, the links it inherits from the primary). Its links are taken back, so memory and disk both have the node without them. The create was told (EV_TREEDB_NODE_CREATED), and an ERROR says what is left: “Node created, but its links cannot be saved (autolink): the node stays without them”.

• update + autolink: the node, its fields and its links are as the disk has them, in memory and on disk. One exception: the bytes of a file column of the record are stored BEFORE the write opens, as a write of their own. The blob and its __assets__ node stay on disk, and their EV_TREEDB_NODE_CREATED (or EV_TREEDB_NODE_UPDATED, for a new name of bytes already stored) is told. The node does not link them.

In 7.25.4 this write was three calls, and a failed save took nothing back: memory kept the fields and the links until the next reload, and EV_TREEDB_NODE_LINKED had been told. Write it again once the cause is fixed: the same update-node with autolink converges.

/*  the store of `users` cannot be written (a full disk)  */
json_t *node = gobj_update_node(gobj_node, "users",
    json_pack("{s:s, s:s, s:[s]}",
        "id", "alice",
        "username", "ALICE-NEW",
        "departments", "departments^direction^users"
    ),
    json_pack("{s:b}", "autolink", 1),
    src
);
/*  node == NULL; alice keeps her old username and links, in memory as on
 *  disk; direction does not hook her; no event was published            */

EV_TREEDB_UPDATE_NODE is an input event for gobjs of the SAME yuno (gobj_send_event()), with no permission asked. It is not a public event since after 7.25.3: C_IEVENT_CLI delivered a public event from its peer to the local service the event names, so the other end of an outbound session could write the treedb.

Example: a user whose only role on treedb_devices has "permission": "read" gets nodes and node, and is refused link-nodes:

ycommand -c 'command-yuno id=<yuno> service=treedb_devices command=link-nodes parent_ref=places^p1^devices child_ref=devices^d1'
# -403 No permission to 'update' in service 'treedb_devices'

What a command names is the peer’s data¶

A topic, a hook, a link, a parent ref or a record that a command names comes from the peer, and a mistake in it is the peer’s, not a broken contract of the yuno. C_NODE refuses it after the permission check (a peer without the permission learns nothing about which topics exist) with a WARNING (msgset: MSGSET_PARAMETER; a name that is not yet known to be valid is capped at 64 bytes, never in the middle of a character) and answers -1. The gobj_*_node() methods and the treedb keep their ERROR for the same mistakes when the caller is C code -- including a subscriber that writes from its EV_TREEDB_NODE_* handler during the command, or from the callback of an rt list of the topic (C_TRANGER’s open-rt): the treedb publishes and appends outside the peer’s window -- because for a C caller it is a broken contract.

What only the treedb can judge -- a field required or of the wrong type, an fkey that cannot be linked, a link that would close a cycle or links a node to itself, an unlink of a child that does not hang from that parent, a create of an id that exists or holds a path metacharacter, a value not in its enum, a delete of a node a snapshot holds, a seed link the command would cut, a time in the topic’s tkey that the md2 row of the record cannot keep -- is logged by the treedb (or by the method) with its own message, as a WARNING without a stack: the command marks the record as a peer’s around the call (treedb_set_peer_data(), since SDK 7.27.0), and answers -1 with “cannot ... (see the log)”.

# `users` links users, not departments (SDK 7.27.0+; before it an ERROR of the treedb)
command-yuno id=<id> service=<treedb> command=link-nodes parent_ref=departments^top^users child_ref=departments^dev
# -> -1 <role^name>: Hook 'users' does not link topic 'departments'

# an update of a node that is not there: add options='{"create":1}' to make it
command-yuno id=<id> service=<treedb> command=update-node topic_name=departments record='{"id":"nosuch","name":"Nobody"}'
# -> -1 <role^name>: Node not found: 'nosuch' of topic 'departments'

# a link that closes a cycle: dev hangs from top (a WARNING of the treedb, no stack)
command-yuno id=<id> service=<treedb> command=link-nodes parent_ref=departments^dev^departments child_ref=departments^top
# -> -1 <role^name>: cannot link 'departments^top' to 'departments^dev^departments' (see the log)

The time of the tkey must fit the record. The tkey of a topic is the __tm__ of each record’s md2 row, which keeps 44 bits (TRANGER2_MAX_TIME). A topic made by a peer (create-topic ... topic_tkey=when) can have a string tkey, read as a date, and a date has no year limit: "@99999999999999 +0000" was a node whose __tm__ was not its date, cut without a word. create-node and update-node refuse it before anything is written (new after 7.26.9), with “Node refused: the time of its tkey does not fit a record (negative, or past 44 bits)” (a WARNING from a peer; the same gobj_create_node() / gobj_update_node() of C code is an ERROR with a stack, and is refused too). An update that does not write the tkey keeps the time the node had, and a link, an unlink or a snapshot of a node that already holds such a time is not refused (see treedb_create_node()). For example, with the string tkey when:

# events: topic_tkey=when, a string column
command-yuno id=<id> service=<treedb> command=create-node topic_name=events record='{"id":"e1","when":"@99999999999999 +0000"}'
# -> -1 <role^name>: cannot create the node of topic 'events' (see the log)
command-yuno id=<id> service=<treedb> command=create-node topic_name=events record='{"id":"e2","when":"@1760000000 +0000"}'
# -> 0 <role^name>: Node created, 'e2' of topic 'events'
command-yuno id=<id> service=<treedb> command=update-node topic_name=events record='{"id":"e2","when":"@99999999999999 +0000"}'
# -> -1 <role^name>: cannot update the node of topic 'events' (see the log)

A tkey the clock writes always fits. The clock overwrites a column on every create and every update only when it is of type integer and flagged now: it stamps the time in seconds, and the value the caller sends is ignored. So such a tkey is never refused. Not every topic has one: a topic can have no tkey ('tkey': '', the default of create-topic), and a now on a column of another type stamps nothing (a string tkey flagged now keeps the caller’s text, and is checked as above). An integer column flagged time without now is not one either: it gets the clock only at a create that sends no value for it and whose column has no default, an update that does not send it leaves it as it is, and a value that is sent is kept and checked as above. A topic whose tkey the clock writes, in the form of a C schema literal (as users_accesses of treedb_authzs):

{
    'id': 'accesses',
    'pkey': 'id',
    'tkey': 'tm',
    'system_flag': 'sf_string_key',
    'topic_version': '1',
    'cols': {
        'id': {'header': 'User', 'type': 'string',  'flag': ['persistent', 'required']},
        'tm': {'header': 'Time', 'type': 'integer', 'flag': ['time', 'now', 'persistent']}
    }
}

# accesses: tm is stamped by the clock; the value sent is ignored, so it is never refused
command-yuno id=<id> service=<treedb> command=create-node topic_name=accesses record='{"id":"a1","tm":17592186044416}'
# -> 0 <role^name>: Node created, 'a1' of topic 'accesses'

A tkey flagged time without now, as 'born': {'header': 'Born', 'type': 'integer', 'flag': ['time', 'persistent']} of a topic births with 'tkey': 'born', gets the clock only when the create sends nothing:

command-yuno id=<id> service=<treedb> command=create-node topic_name=births record='{"id":"b1"}'
# -> 0 <role^name>: Node created, 'b1' of topic 'births'      (born: the time of the create)
command-yuno id=<id> service=<treedb> command=create-node topic_name=births record='{"id":"b2","born":17592186044416}'
# -> -1 <role^name>: cannot create the node of topic 'births' (see the log)

With a default on that column, 'born': {'header': 'Born', 'type': 'integer', 'flag': ['time', 'persistent'], 'default': 0}, the create of b1 stores 0, not the clock: the default is the value the create did not send.

File columns: __assets__¶

A treedb node often owns something that is not JSON — a photo, a plan, a signed pdf. Those bytes cannot go in the treedb: it is held in memory and timeranger2 rewrites the whole record on every update, so a 40 KB photo would be rewritten every time its node changed state and would ride along in every page of nodes. Measured on one census: 12 134 blobs, 346 MB on disk, would be ~460 MB of RAM for the life of the yuno.

So you mark a column file, and treedb gives you a pseudo-filesystem: you hand it a file, you get it back; the index lives in memory and the content on disk. Design note: DESIGN-treedb-files.md.

'foto': {'header': 'Photo', 'type': 'string', 'flag': ['fkey', 'file']}

• The column is an fkey into __assets__, a system topic every treedb creates at open next to __snaps__ and __graphs__ (shown in system mode only). Its rows are the index entries: id (the sha256 of the content), content_type, size, t, original_name (the only writable one), uploaded_by. Its hooks are derived: for every column C of topic T flagged file, __assets__ gains as_<T>_<C> -> {T: C} in memory, never written to topic_cols.json. The host declares nothing but the column, and nothing about __assets__ is ever versioned. The derivation follows the schema at run time: create-topic and delete-topic are live commands, so a topic added while the yuno runs gets its hook, and one deleted takes its hook — and the children it held — away with it.

• The bytes live under the treedb, at <treedb dir>/.blobs/ab/cd/<sha256>.<ext>, so a cp -a of the treedb directory carries the nodes and their bytes.

• The bytes ride beside the record, never inside the column, in a __files__ manifest keyed by column, consumed at the door. A browser sends content64; a C caller (command or EV_TREEDB_UPDATE_NODE alike) puts a real gbuffer in the kw and the manifest says which slice is whose (offset, size) — a kw holds ONE binary field, so one buffer carries every file of the record:

{"topic_name": "devices",
 "record": {"id": "E22000041",
            "foto": "",
            "__files__": {"foto": {"content64": "...", "original_name": "E22000041.jpg",
                                   "content_type": "image/jpeg"}}},
 "options": {"create": 1, "autolink": 1}}

• Treedb re-hashes what arrives. A client may put the sha256 in the column up front, ask node whether that asset exists and skip the bytes if it does (a census reload then sends ~0 instead of 346 MB) — but the id is a claim, never an authority: a wrong id with good bytes is refused, and a bare id of an asset nobody stored is refused. A bare id of an existing asset links it. A column may also arrive holding the full reference, and then it must be exactly __assets__^<id>^as_<T>_<C> of that very column: the link is made by the hook the value names, so one naming another file column’s hook would store the file into that other column.

• Size and type are checked at the door, on the bytes. The size is checked on the base64 before decoding; the type is sniffed from the first bytes and the declared one must agree — a png called image/jpeg is refused, and an svg called image/png is refused, which is the case the allowlist exists for. Two levels: the treedb’s ceiling (files_max_size, files_content_types) and the column’s policy, in its properties ({'max_size': 4096, 'content_types': ['application/pdf']}), which narrows the ceiling and never raises it. The ceiling itself sits behind the transport’s: the message was accepted whole and parsed before treedb saw it, so keep files_max_size under the transport’s max_pkt_size.

• Three writes, in order: the blob, the __assets__ node, the host record with its link. Interrupted early it leaves an orphan blob or an orphan index node, which gc-assets takes; never a link to nothing. A create-node whose file column cannot be linked is undone, so the answer never says yes over a record with an empty column.

• The write path links the file column itself, autolink or not. An ordinary fkey moves only through link-nodes or an autolink; a file column is edited by handing over a file, and the link is part of it: create-node and update-node link what the column names, "" unlinks, and a column the record does not carry is left alone. The autolink in the example above is for the OTHER fkeys of the record, not for foto.

• A second arrival of the same bytes is an update of the asset node, so the history of __assets__ says every name a file arrived under (a manifest that carries no original_name, or the name already stored, says nothing new about the file and appends nothing). The blob is written once, and the first arrival names it for ever: the extension is part of the served path and the URL is cached for ever, so a later arrival that declares another member of the same container (audio/mp4 for what was stored as video/mp4) keeps the stored content_type and says so in the log.

• gc-assets reads the snapshots, exactly. shoot-snap skips every __ topic, so an asset node never carries a tag and the tag guard never protects it: the collector walks the instances on disk of every topic with a file column and keeps what an ACTIVATION would load — per key, the newest instance under each existing snap’s tag, and only that one (a save is untagged since 7.24.0, so that instance is the record the snap SHOT, and a node that moves on after the shot does not drag the snap’s hold along). A snapshot holds bytes alive; deleting the snap frees them, and the delete is delete-node on its __snaps__ row (there is no delete-snap command). A treedb with no snapshot does not walk. It also takes the bytes with no row — what an interrupted write leaves, and the .tmp of one that never reached its rename — because every other reader of the store goes through the rows. .blobs is the tranger’s, so what counts as named is the union of every treedb’s __assets__.

• Deleting an asset node deletes its bytes. Refused while a node links it, like any parent with children — and refused, force included, while a snapshot links it: delete-node on an __assets__ row runs the same walk gc-assets does, because the ordinary tag guard never fires for an asset. force means “unlink the children”, never “ignore what a snapshot needs”, and no key of options skips the walk: the gc skips it through a private entry, not through anything the wire can spell. On a tranger that hosts more than one treedb, __assets__ and its bytes are shared: both gc-assets and delete-node read every treedb’s links, and an asset another treedb of the tranger links is refused, force or not.

• A file column must be ['fkey','file'] on a string, and that is checked in three places: at open (fatal), by create-topic with the yuno running (the topic is refused, the answer says why), and by the write path (the write is refused). A column that is file and not fkey would store its bytes into a column nothing links, and gc-assets would take them.

• ⚠️ A non-master replica gets the index and not the bytes. The watcher replicates a topic’s files, so a replica has every row of __assets__ and none of the .blobs behind them: get-asset answers “asset has no bytes on disk” there unless a web server in front is serving a copy of the blob directory. Getting the bytes to a replica is not solved — plan for it, or serve the assets from the master. The same holds for export-db / import-db: they carry the rows of __assets__ and not the .blobs.

Serving the bytes to a browser is C_ASSETS’ job.

C_RESOURCE2¶

Simple resource persistence — stores each resource as a flat JSON file.

Key attributes¶

C_ASSETS¶

The way out of the bytes a treedb keeps for its file columns: a signed URL a web server checks by itself, or the bytes inline when there is no web server in front. Storing is treedb’s (File columns): file columns, __assets__, and the import-assets / gc-assets commands of C_NODE. This gclass only publishes what is stored, and is one command.

The asset id is the sha256 of its content, so a served URL means the same bytes for ever and can be cached for ever. get-asset takes the asset id and nothing else — not a node plus a column: keyed by node, the same URL would mean different bytes the moment the node was relinked.

Key attributes¶

Commands¶

Gated by the read authz of the service.

get-asset takes asset_id, not id. command-yuno hands its whole kw to gobj_list_nodes() as the filter that picks the yuno, so a parameter named like a field of the yuno record becomes a filter on that field: an id of a sha256 matches no yuno and the answer is “Yuno not found” — which names the yuno and never the parameter. The bare id still works for a caller that never crosses the agent.

Two ways out to a browser, and the service picks¶

get-asset answers in one of two shapes:

{"mode": "url",    "url": "/media/ab/cd/<id>.jpg?e=<expires>&s=<token>"}
{"mode": "inline", "content_type": "image/jpeg", "content64": "..."}

It signs a URL when public_url and sign_secret are both configured, and answers inline when they are not. So the caller has one code path, and a node with no web server in front of it still shows its images instead of showing nothing.

The signed form reproduces, byte for byte, what this nginx block hashes. Do not use /assets/: a Vite SPA on the same vhost already owns that prefix for its content-hashed bundles, and the two locations would fight over it. The alias is the treedb’s own blob directory.

location /media/ {
    secure_link      $arg_s,$arg_e;
    secure_link_md5  "$secure_link_expires$uri <sign_secret>";
    if ($secure_link = "")  { return 403; }   # bad signature
    if ($secure_link = "0") { return 410; }   # expired
    alias <store>/<realm>/treedb_<name>/.blobs/;
    expires max;    # the name IS the hash: it can never go stale
    access_log off;
}

The client address is deliberately not in the signature: it would tie the URL to one IP and break every phone that changes network mid-session. The short lifetime is what limits a leaked URL.

secure_link needs nginx built --with-http_secure_link_module. Yuneta’s nginx and openresty are, but a node still running an older build must serve inline until its web server is replaced.

tests/c/c_assets covers the whole round trip: a record with its bytes beside it through update-node, both doors (content64 and a gbuffer of two slices), get-asset inline and signed, import-assets with hostile paths (.., absolute, a symlink out of the root), and gc-assets.