arachnespace::arachne Class Reference

Accumulates entity IDs into per-kind batches and organizes groups. More...

#include <include/arachne.hpp>

Collaboration diagram for arachnespace::arachne:

Private Member Functions
void	select_group (std::string name)
	Select an existing group or create it on demand.
bool	enqueue (std::string_view id, corespace::entity_kind kind, bool interactive) const
	Decide whether an entity should be enqueued for fetching.
bool	touch_entity (const std::string &id_with_prefix) noexcept
	Increment the touch counter for a single full ID (prefix REQUIRED).
size_t	add_entity (const std::string &id_with_prefix, bool force=false, std::string name="")
	Enqueue a full (prefixed) ID string and add it to a group.

Static Private Member Functions
static bool	ask_update (std::string_view id, corespace::entity_kind kind, std::chrono::milliseconds age)
	Placeholder for interactive staleness confirmation.

Private Attributes
std::array< std::unordered_set< std::string >, batched_kind_count >	main_batches
std::array< std::unordered_set< std::string >, batched_kind_count >	extra_batches
std::unordered_map< std::string, std::unordered_set< std::string > >	groups
std::unordered_map< std::string, int >	candidates
const size_t	batch_threshold = 50
	Typical unauthenticated entity-per-request cap.
const int	candidates_threshold = 50
	Intentional high bar for curiosity-driven candidates.
std::string	current_group
std::chrono::milliseconds	staleness_threshold = 24h
corespace::interface	ui = corespace::interface::command_line
pheidippides	phe_client

Public API
bool	new_group (std::string name="")
	Create or select a group and make it current.
size_t	add_ids (std::span< const int > ids, corespace::entity_kind kind, std::string name="")
	Enqueue numeric IDs with a given kind and add them to a group.
int	touch_ids (std::span< const int > ids, corespace::entity_kind kind)
	Batch variant of touch for numeric IDs.
bool	flush (corespace::entity_kind kind=corespace::entity_kind::any)
	Flush (send) up to batch_threshold entities of a specific kind.
int	queue_size (corespace::entity_kind kind) const noexcept
	Get the number of queued (pending) entities tracked in the main batch containers.
static std::string	entity_root (const std::string &id)
	Extract the lexeme root from a full ID string.
static corespace::entity_kind	identify (const std::string &entity) noexcept
	Determine the kind of a full ID string.
static bool	parse_id (const std::string &entity, size_t &pos, int &id)
	Parse a full ID string and extract the numeric portion.
static std::string	normalize (int id, corespace::entity_kind kind)
	Normalize a numeric ID with the given kind to a prefixed string.

Detailed Description

Accumulates entity IDs into per-kind batches and organizes groups.

Invariants:

• Queues store normalized ID strings per kind ("Q123", "P45", "L7", "M9", "E2", "L7-F1", "L7-S2").
• For numeric add/touch with kind = form or sense, normalization produces "L<id>" (warning), because numeric IDs for forms/senses are not representable; string APIs keep the exact ID.
• Deduplication is by string identity in the respective containers.

Definition at line 47 of file arachne.hpp.

Member Function Documentation

add_entity()

size_t arachnespace::arachne::add_entity ( const std::string & id_with_prefix, bool force = false, std::string name = "" )

private

Enqueue a full (prefixed) ID string and add it to a group.

The ID must include its prefix (e.g., "Q123", "L77-F2"). Validation is performed via identify(). Invalid IDs cause an exception. For "L...-F..."/"L...-S...", the group receives the verbatim string while the batch queue stores the lexeme root ("L...") so fetches target the parent lexeme.

Parameters

id_with_prefix	Full ID with prefix.
force	If true, bypass freshness/existence checks and enqueue anyway.
name	Group name; empty targets the current/anonymous group (auto-created if needed).

Returns

The resulting size of the target group after insertion.

Exceptions

std::invalid_argument

if the ID is invalid or has an unknown prefix.

Definition at line 235 of file arachne.cpp.

  {
    const std::string canonical = entity_root(id_with_prefix);
    select_group(std::move(name));
    auto& group = groups[current_group];
    group.insert(id_with_prefix);
    if (corespace::entity_kind kind = identify(canonical); force
        || enqueue(canonical, kind, ui == corespace::interface::command_line)) {
        auto& pool = main_batches[static_cast<size_t>(kind)];
        pool.insert(canonical);
        if (pool.size() >= batch_threshold) {
            flush(kind);
        }
    }
    return group.size();
}

References entity_root(), flush(), identify(), and select_group().

Here is the call graph for this function:

add_ids()

size_t arachnespace::arachne::add_ids	(	std::span< const int >	ids,
		corespace::entity_kind	kind,
		std::string	name = "" )

Enqueue numeric IDs with a given kind and add them to a group.

Numeric IDs are normalized by adding the kind prefix.

• If kind is form or sense, normalization maps to the lexeme prefix ("L<id>"); no warning is emitted yet (logging TODO).
• Freshness checks are stubbed; the helper enqueue always asks for a fetch, and the underlying sets deduplicate repeated IDs automatically.

Parameters

ids	Span of numeric IDs.
kind	Entity kind (must NOT be any/unknown).
name	Group name; empty targets the current/anonymous group (auto-created if needed).

Returns

The resulting size of the target group after insertions.

Exceptions

std::invalid_argument

if kind is any/unknown.

Definition at line 42 of file arachne.cpp.

  {
    if (kind == corespace::entity_kind::any
        || kind == corespace::entity_kind::unknown) {
        throw std::invalid_argument("unknown kind of numeric IDs");
    }
    select_group(std::move(name));
    size_t last_size = groups[current_group].size();
    for (const int id : ids) {
        std::string id_with_prefix = normalize(id, kind);
        last_size = add_entity(id_with_prefix, false, current_group);
    }
    return last_size;
}

References corespace::any, select_group(), and corespace::unknown.

Here is the call graph for this function:

ask_update()

bool arachnespace::arachne::ask_update ( std::string_view id, corespace::entity_kind kind, std::chrono::milliseconds age )

staticprivate

Placeholder for interactive staleness confirmation.

The current implementation is non-interactive and always returns false. A future version is expected to prompt the user when cached data is stale and return the user's decision.

Parameters

id	Entity identifier under consideration.
kind	Detected kind of the entity.
age	Age of the cached entry.

Returns

Currently always false; future behavior should reflect user confirmation.

Definition at line 194 of file arachne.cpp.

  {
    // UI/UX: todo: ask user if update is needed
    return false;
}

enqueue()

bool arachnespace::arachne::enqueue ( std::string_view id, corespace::entity_kind kind, bool interactive ) const

private

Decide whether an entity should be enqueued for fetching.

This placeholder implementation always returns true, effectively requesting a fetch for every entity. The expected behavior is to consult storage state (exist, last) and return true only when an update is required.

Parameters

id	Canonical identifier (e.g., "Q123" or "L7").
kind	Entity kind (lexeme for forms/senses).

Returns

true if the caller should enqueue the entity; placeholder always true.

Definition at line 201 of file arachne.cpp.

        {
    // ariadne.entity_status(id)
    auto [exist, last] = std::pair<bool, long long>(false, -1);
    if (!exist || last < 0) {
        return true;
    }
    const auto now_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
                            std::chrono::system_clock::now().time_since_epoch()
    )
                            .count();
    const auto age = std::chrono::milliseconds { now_ms - last };
    if (age > staleness_threshold) {
        return true;
    }
    if (interactive) {
        return ask_update(id, kind, age);
    }
    return false;
}

entity_root()

std::string arachnespace::arachne::entity_root ( const std::string & id )

static

Extract the lexeme root from a full ID string.

For IDs beginning with "L" followed by digits, returns "L<digits>". For other prefixes or malformed strings, returns an empty string.

Parameters

id	Identifier to inspect (e.g., "L7-F1").

Returns

Lexeme root ("L7") or empty on failure.

Definition at line 74 of file arachne.cpp.

                                                  {
    const corespace::entity_kind kind = identify(id);
    if (kind == corespace::entity_kind::any
        || kind == corespace::entity_kind::unknown) {
        throw std::invalid_argument("invalid or unknown entity kind");
    }
 
    if (kind == corespace::entity_kind::form
        || kind == corespace::entity_kind::sense) {
        if (id.size() < 2 || id.front() != 'L') {
            throw std::invalid_argument(
                "bad root-lexeme prefix of the entity: " + id
            );
        }
        int val {};
        if (size_t pos = 1; !parse_id(id, pos, val)) {
            throw std::invalid_argument(
                "bad numeric identifier of the entity: " + id
            );
        }
        return "L" + std::to_string(val);
    }
    return id;
}

References corespace::any, corespace::form, identify(), corespace::sense, and corespace::unknown.

Referenced by add_entity(), and touch_entity().

Here is the call graph for this function:

Here is the caller graph for this function:

flush()

bool arachnespace::arachne::flush

(

corespace::entity_kind

kind = corespace::entity_kind::any

)

Flush (send) up to batch_threshold entities of a specific kind.

For kind != any, attempts a single-batch flush for that kind (up to the threshold). For kind == any, a round-robin strategy over batchable kinds is used.

Parameters

kind	Entity kind selector or entity_kind::any.

Returns

true if at least one entity was flushed; false otherwise.

Definition at line 99 of file arachne.cpp.

                                             {
    const auto& batch = main_batches[static_cast<size_t>(kind)];
    const size_t size = batch.size();
    auto data = phe_client.fetch_json(batch, kind);
    // ariadne.store(data);
    return size > batch.size();
}

Referenced by add_entity().

Here is the caller graph for this function:

identify()

corespace::entity_kind arachnespace::arachne::identify ( const std::string & entity )

staticnoexcept

Determine the kind of a full ID string.

Accepts prefixed IDs (e.g., "Q123", "L77-F2"). Returns unknown if the string is not a valid ID. The function does not throw.

Parameters

entity

Full ID with prefix.

Returns

Detected kind (may be unknown).

Definition at line 122 of file arachne.cpp.

                                                                       {
    if (entity.size() < 2) {
        return corespace::entity_kind::unknown;
    }
    size_t pos = 0;
    size_t kind = prefixes.find(entity[pos++]);
    int id {};
    if (kind == std::string::npos || !parse_id(entity, pos, id)) {
        return corespace::entity_kind::unknown;
    }
    if (pos == entity.size()) {
        return static_cast<corespace::entity_kind>(kind);
    }
    if (kind != static_cast<size_t>(corespace::entity_kind::lexeme)
        || pos >= entity.size() || entity[pos++] != '-'
        || pos >= entity.size()) {
        return corespace::entity_kind::unknown;
    }
    const char tag = entity[pos++];
    if (tag != 'F' && tag != 'S' || !parse_id(entity, pos, id)
        || pos != entity.size()) {
        return corespace::entity_kind::unknown;
    }
    return tag == 'F' ? corespace::entity_kind::form
                      : corespace::entity_kind::sense;
}

References corespace::form, corespace::lexeme, arachnespace::prefixes, corespace::sense, and corespace::unknown.

Referenced by add_entity(), entity_root(), and touch_entity().

Here is the caller graph for this function:

new_group()

bool arachnespace::arachne::new_group

(

std::string

name = ""

)

Create or select a group and make it current.

If name is empty, creates a new anonymous group with a random name and makes it current. If name exists, it becomes current but is NOT cleared. If it doesn't exist, the group is created and then selected.

Parameters

name	Group name or empty for an anonymous group.

Returns

true if a new group was created; false if the group already existed.

Note

The current group's name is intentionally not exposed; anonymous groups cannot be addressed explicitly.

Definition at line 31 of file arachne.cpp.

                                      {
    if (name.empty()) {
        do {
            name = "g_" + corespace::random_hex(8);
        } while (groups.contains(name));
    }
    auto [it, inserted] = groups.try_emplace(name);
    current_group = it->first;
    return inserted;
}

References current_group, and corespace::random_hex().

Referenced by select_group().

Here is the call graph for this function:

Here is the caller graph for this function:

normalize()

std::string arachnespace::arachne::normalize ( int id, corespace::entity_kind kind )

static

Normalize a numeric ID with the given kind to a prefixed string.

Examples:

• (123, item) -> "Q123"
• (45, property) -> "P45"
• (7, lexeme) -> "L7"
• (9, mediainfo) -> "M9"
• (2, entity_schema) -> "E2"
• (7, form) -> "L7" (mapped to lexeme)
• (7, sense) -> "L7" (mapped to lexeme)

Parameters

id	Numeric identifier.
kind	Kind to prefix with (must not be any/unknown).

Returns

Prefixed ID string.

Exceptions

std::invalid_argument

if id is negative or kind is any/unknown.

Note

Form and sense identifiers are currently coerced to the lexeme prefix without emitting diagnostics; logging is a planned enhancement.

Definition at line 165 of file arachne.cpp.

                                                                {
    if (id < 0) {
        throw std::invalid_argument("normalize: id must be non-negative");
    }
    if (kind == corespace::entity_kind::any
        || kind == corespace::entity_kind::unknown) {
        throw std::invalid_argument(
            "normalize: kind must be a concrete, known entity kind"
        );
    }
    auto idx = static_cast<std::size_t>(kind);
    if (idx >= static_cast<size_t>(corespace::entity_kind::form)) {
        // Numeric Form/Sense are not representable; map to lexeme.
        // TODO: emit warning via logging sink.
        idx = static_cast<size_t>(corespace::entity_kind::lexeme);
    }
    return prefixes[idx] + std::to_string(id);
}

References corespace::any, corespace::form, corespace::lexeme, and corespace::unknown.

parse_id()

bool arachnespace::arachne::parse_id ( const std::string & entity, size_t & pos, int & id )

static

Parse a full ID string and extract the numeric portion.

Parameters

entity	Full ID (e.g., "Q123", "L7-F1", "L7-S2").
pos	In/out index of the first digit within entity. On success the index is advanced past the number.
id	Out parameter for the parsed integer portion.

Returns

true on successful parse; false otherwise. Never throws.

Definition at line 149 of file arachne.cpp.

                                                                    {
    id = 0;
    size_t len = 0;
    try {
        id = std::stoi(entity.substr(pos), &len);
    } catch (...) {
        return false;
    }
    if (id < 0 || len == 0 || std::to_string(id).size() != len) {
        return false;
    }
    pos += len;
    return true;
}

queue_size()

int arachnespace::arachne::queue_size ( corespace::entity_kind kind ) const

noexcept

Get the number of queued (pending) entities tracked in the main batch containers.

Parameters

kind	Specific kind, or entity_kind::any to return the sum across all batchable kinds.

Returns

Count of queued entities for the requested kind. Values are narrowed to int via static_cast and therefore inherit the implementation-defined behavior of signed narrowing conversions when the count exceeds INT_MAX.

Definition at line 107 of file arachne.cpp.

                                                                      {
    if (kind == corespace::entity_kind::any) {
        std::size_t sum = 0;
        for (const auto& batch : main_batches) {
            sum += batch.size();
        }
        return static_cast<int>(sum);
    }
    const auto idx = static_cast<std::size_t>(kind);
    if (idx >= main_batches.size()) {
        return 0;
    }
    return static_cast<int>(main_batches[idx].size());
}

References corespace::any.

select_group()

void arachnespace::arachne::select_group ( std::string name )

private

Select an existing group or create it on demand.

An empty name selects/creates the anonymous group. A non-empty name is delegated to new_group, which creates the group if necessary.

Parameters

name	Group name to activate; empty targets the anonymous group.

Definition at line 184 of file arachne.cpp.

                                         {
    if (name.empty()) {
        if (current_group.empty()) {
            new_group();
        }
        return;
    }
    new_group(std::move(name));
}

References current_group, and new_group().

Referenced by add_entity(), and add_ids().

Here is the call graph for this function:

Here is the caller graph for this function:

touch_entity()

bool arachnespace::arachne::touch_entity ( const std::string & id_with_prefix )

privatenoexcept

Increment the touch counter for a single full ID (prefix REQUIRED).

If the entity is already queued or already has data, returns false (no increment). If the counter reaches candidates_threshold and the entity is not queued, it is moved into the queue. For "L…-F…"/"L…-S…", the exact ID is enqueued (no mapping).

Parameters

id_with_prefix

Full ID with prefix.

Returns

true if the counter was incremented; false otherwise.

Definition at line 224 of file arachne.cpp.

                                                                   {
    candidates[id_with_prefix]++;
    if (candidates[id_with_prefix] >= candidates_threshold) {
        const std::string canonical = entity_root(id_with_prefix);
        corespace::entity_kind kind = identify(canonical);
        extra_batches[static_cast<size_t>(kind)].insert(canonical);
        return true;
    }
    return false;
}

References entity_root(), and identify().

Here is the call graph for this function:

touch_ids()

int arachnespace::arachne::touch_ids	(	std::span< const int >	ids,
		corespace::entity_kind	kind )

Batch variant of touch for numeric IDs.

Each numeric ID is normalized using kind. If kind is form/sense, a warning is recorded and normalization yields "L<id>" (lexeme).

Parameters

ids	Span of numeric IDs.
kind	Normalization kind (must not be any/unknown).

Returns

The number of entities for which touch_entity() returned true.

Exceptions

std::invalid_argument

if kind is any/unknown.

Definition at line 59 of file arachne.cpp.

  {
    if (kind == corespace::entity_kind::any
        || kind == corespace::entity_kind::unknown) {
        throw std::invalid_argument("unknown kind of numeric IDs");
    }
    int added = 0;
    for (const int id : ids) {
        std::string id_with_prefix = normalize(id, kind);
        added += touch_entity(id_with_prefix);
    }
    return added;
}

References corespace::any, and corespace::unknown.

Member Data Documentation

batch_threshold

const size_t arachnespace::arachne::batch_threshold = 50

private

Typical unauthenticated entity-per-request cap.

Definition at line 283 of file arachne.hpp.

candidates

std::unordered_map<std::string, int> arachnespace::arachne::candidates

private

Definition at line 280 of file arachne.hpp.

candidates_threshold

const int arachnespace::arachne::candidates_threshold = 50

private

Intentional high bar for curiosity-driven candidates.

Definition at line 285 of file arachne.hpp.

current_group

std::string arachnespace::arachne::current_group

private

Definition at line 290 of file arachne.hpp.

Referenced by new_group(), and select_group().

extra_batches

std::array<std::unordered_set<std::string>, batched_kind_count> arachnespace::arachne::extra_batches

private

Definition at line 273 of file arachne.hpp.

groups

std::unordered_map<std::string, std::unordered_set<std::string> > arachnespace::arachne::groups

private

Definition at line 277 of file arachne.hpp.

main_batches

std::array<std::unordered_set<std::string>, batched_kind_count> arachnespace::arachne::main_batches

private

Definition at line 271 of file arachne.hpp.

phe_client

pheidippides arachnespace::arachne::phe_client

private

Definition at line 293 of file arachne.hpp.

staleness_threshold

std::chrono::milliseconds arachnespace::arachne::staleness_threshold = 24h

private

Definition at line 291 of file arachne.hpp.

ui

corespace::interface arachnespace::arachne::ui = corespace::interface::command_line

private

Definition at line 292 of file arachne.hpp.

---

The documentation for this class was generated from the following files:

• arachne.hpp
• arachne.cpp