Commit Graph

109 Commits

Author SHA1 Message Date
Guillem Arias
af647a9cfc feat(beta-gating): beta_gated_nav_item helper auto-marks gated entries
Wraps the conditional + dot wiring into a single call so adding a new
beta nav entry doesn't require remembering to set `beta: true` by hand
or duplicating the `beta_features_enabled?` check. Naming mirrors the
existing `BetaGateable` concern.
2026-05-18 20:43:02 +02:00
Guillem Arias
7b4cee60aa docs(beta-gating): document main-nav dot marker via beta: local
The nav-item partial already supports a `beta: true` local that overlays
the DS::Pill dot on the icon, but the gating guide didn't show how to
wire a gated nav entry through it. Add a short "Gating the main nav"
section with the compact-array pattern, and mention the flag in the GA
removal checklist.
2026-05-18 20:37:46 +02:00
Guillem Arias
ac23521c0a Merge remote-tracking branch 'origin/main' into feat/goals-v2-architecture 2026-05-18 20:09:48 +02:00
Guillem Arias Fauste
5249842c76 feat: beta features toggle + Beta pill primitive (#1829)
* feat: beta features toggle + Beta pill primitive

Adds the infrastructure for self-service beta opt-in. No call sites yet:
this PR is meant to land first so feature PRs (Goals, etc.) can ship
behind the gate incrementally.

User opts in via a single toggle at the bottom of Settings → Preferences.
The flag persists in the existing `users.preferences` JSONB column under
`beta_features_enabled` — same shape as `dashboard_two_column` and
`show_split_grouped`, so no migration is needed.

Controllers gate a beta feature by adding `before_action
:require_beta_features!` from the new `BetaGateable` concern (included in
ApplicationController). Views use the `beta_features_enabled?` helper to
hide / show nav items, banners, etc. Logged-out callers always return
false.

Ships `DS::BetaPill`, a small inline marker for tagging features as
Beta / Canary in nav, headers, and lists. Five tones (violet by default,
indigo, fuchsia, amber, gray) map to existing Sure color tokens — no raw
hex. Three styles (soft / filled / outline) and two sizes (sm / md) cover
the surfaces in the design handoff. The `dot_only:` mode renders just
the colored dot for use on a collapsed sidebar.

* review: rename to DS::Pill, fix CR/Codex nits, add tests

CodeRabbit + Codex review feedback:

- Rename DS::BetaPill → DS::Pill. The component was already generic in
  shape (tones, styles, sizes); the name was misleading scope. "Beta"
  becomes the default label (still i18n-driven). Goals' StatusPill can
  later refactor onto this primitive without a third pill.
- Localize the default pill label via i18n (`ds.pill.default_label`)
  instead of hard-coding English.
- Add role="img" to the dot-only span so the aria-label is consistently
  exposed to assistive tech.
- Wrap the Preferences toggle row in <label for="…"> so the title and
  description become an honest click target for the toggle (matches the
  cursor-pointer affordance).
- Drop arbitrary Tailwind values (py-[3px], gap-[5px], tracking-[…]) in
  favor of scale tokens. text-[10/11px] stays because the pill is
  intentionally sub-12px (Sure's smallest scale token is text-xs / 12px)
  to read as a marker, not a label.
- Add User#beta_features_enabled? predicate tests covering default-off,
  explicit-true, and non-boolean truthy values.

Won't fix:
- Palette refs (`--color-violet-*` etc.). Sure has no semantic Beta/
  Canary tokens; introducing them in this PR would be a design-system
  change beyond the scope. The component centralizes palette use in one
  `palette` method, matching the existing pattern in
  Goals::StatusPillComponent.

* review: consistent title fallback in full-pill branch

* docs: how to gate a feature behind the beta toggle

* docs: unwrap doc lines to match existing style

* chore(preview): run Cloudflare PR previews on basic instances (#1831)

* fix(preview): use Rails health endpoint for container ping (#1823)

* fix(preview): use Rails health endpoint for container ping

* fix(preview): point container ping to localhost/up

---------

Co-authored-by: Sure Admin (bot) <sure-admin@splashblot.com>
2026-05-18 20:07:55 +02:00
Sure Admin (bot)
4fd460d551 Add Actual Budget CSV import flow (#1830)
* Add Actual Budget CSV import flow

* Address Actual import review feedback
2026-05-18 18:38:53 +02:00
Guillem Arias
ec385d023c docs(goals): add llm-guide reference for the goals feature
Captures the architecture, key files, data model, status semantics,
pledge match policy, connected-vs-manual account detection, color
map convention, common tasks, and known gotchas. Matches the
existing llm-guides pattern (architecture diagram + file inventory
+ task-oriented sections + reproducible commands).

The doc is forward-looking: it covers how to add a new field to
Goal, a new status branch, a new pledge kind, and how to safely
touch the reconciler. The "Gotchas" section catalogues the
known-incomplete-but-shipping items so a future audit doesn't
re-derive them from scratch.

Demo data regeneration command is included for anyone who needs
to refresh the seed.
2026-05-14 23:07:10 +02:00
Guillem Arias
afc67d07ae chore(goals): drop architecture notes from the repo
Pulls the two early design notes out of git tracking. They covered
the V1 ledger-based design and the engineering mechanics that led
to the V2 rewrite. With V2 shipped, the notes have served their
purpose and the design-of-record now lives in the code + this PR.

Files stay on disk locally (added to .git/info/exclude so future
git status doesn't re-surface them as untracked). Anyone who wants
the V1 reference can pull from the source branch where this work
started.
2026-05-14 22:49:46 +02:00
Guillem Arias Fauste
b32c378a56 Merge branch 'main' into feat/savings-goals
Signed-off-by: Guillem Arias Fauste <accounts@gariasf.com>
2026-05-13 18:22:55 +02:00
Guillem Arias
8b547c1857 docs(goals): expand matrix coverage + ground mechanics in current code
User-facing doc gains explicit 1xM (one goal, multiple accounts),
N goals on shared accounts, and overlap (NxM) sections, plus the
reallocation flow and "why is this goal behind?" diagnostic.

Mechanics doc is rewritten against the actual code on the branch:
file:line citations for current state, accurate corrections to the
prior draft (Sure uses Account AASM status not archived_at, no
Account#balance_at method, balance history via Balance::ChartSeriesBuilder
CTE, Transaction::TRANSFER_KINDS for pace exclusion, advisory-lock
pattern lifted from IdentifyRecurringTransactionsJob, partial-unique
index precedent from entries[external_id, source]), concrete migration
plan with seven steps, surface-by-surface STAY/CHANGE/DELETE verdict
on every component, view, and Stimulus controller, day-one
instrumentation events, and four pre-launch user tests.
2026-05-12 16:59:20 +02:00
Guillem Arias
a6bdfb7335 docs(goals): final architecture cut after 5-iteration expert review
User-facing doc and mechanics companion converged on an account-linked
model with a pledge layer. Surfaces the pledge-with-7-day-match mechanic,
proportional-to-remaining-need split default, "Reserved beyond balance"
framing, in-chart pending segment, two-clock rate limit, archive-in-place
account handling, months-of-runway for open-ended goals, and the
pre-launch user tests + day-one pledge instrumentation.
2026-05-12 16:47:34 +02:00
Guillem Arias
7e464a4b1f docs(goals): split architecture note into user-facing + mechanics
The user-facing note focuses on how a Goal's balance gets computed
and what changes for the user. The mechanics doc covers the schema,
pro-rata math, pace window, and the operational details engineers
need but most readers don't.
2026-05-12 16:30:25 +02:00
Guillem Arias
870fbcd976 docs(goals): add architecture decision discussion note
Captures the open question on the data model behind Goals
(account-linked vs free-form ledger vs tag-based) for community
review before PR #1757 merges.
2026-05-12 16:11:15 +02:00
ghost
325084e342 fix(api): include disabled-account transaction history (#1723)
* fix(api): include disabled-account transaction history

* fix(api): hide pending deletion transaction history
2026-05-12 00:14:13 +02:00
ghost
1fedc43f68 feat(api): add import preflight validation (#1755)
* feat(api): add import preflight validation

* fix(api): harden import preflight validation
2026-05-12 00:00:49 +02:00
ghost
974f55e2d5 feat(api): add transaction idempotency keys (#1729)
* feat(api): add transaction idempotency keys

* fix(api): validate transaction idempotency source

* fix(api): tighten transaction idempotency params
2026-05-11 23:13:24 +02:00
ghost
8abecf8a8d feat(exports): preserve transfer decisions (#1639)
* feat(exports): preserve transfer decisions

* fix(api): apply transfer date filters to both sides

* fix(api): refine transfer decision handling

* fix(api): align transfer decision schemas

* fix(api): use current context for transfer filters

* fix(api): include either side in transfer date filters

* fix(api): deduplicate transfer decision filters

* fix(api): guard transfer decision exports
2026-05-08 23:03:57 +02:00
ghost
45c5284148 feat(api): expose provider connection health (#1636)
* feat(api): expose provider connection health

* fix(api): harden provider health review paths

* fix(api): refine provider health responses

* test(api): align provider health docs key scope

* fix(api): clarify provider connection status

* fix(api): batch provider connection sync status

* fix(api): polish provider connection status review feedback

* fix(api): correct provider connection summaries
2026-05-07 00:42:32 +02:00
GermanDZ
d1081547ec feat(api): allow creating categories via API (#1676)
* feat(api): allow creating categories via API

Adds POST /api/v1/categories so external integrations (e.g. bulk
classification scripts that import already-categorized data from
another system) can create categories without going through the web UI.
Mirrors the existing tags create endpoint: requires the read_write
scope, accepts name/color/icon/parent_id, auto-suggests an icon when
omitted, and rejects parent_ids from other families.

Also adds Minitest behavioural coverage, an rswag docs spec, a
CategoryCreateRequest schema, and regenerates docs/api/openapi.yaml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(api): address review feedback on POST /api/v1/categories

- Re-raise ActionController::ParameterMissing in #create so the
  BaseController rescue_from handles it as a 400 instead of the
  generic 500 from the broad rescue inside the action.
- Add a 403 'insufficient scope' response block to the rswag POST
  example so the generated OpenAPI documents read-only key rejection.
- Switch the new create-action Minitest cases to API key auth via
  X-Api-Key + api_headers (using the existing api_keys fixtures),
  matching the project's API endpoint consistency rule.
- Add Minitest coverage for two more 4xx paths: rejecting third-level
  nesting (parent_id pointing at a depth-2 subcategory) and rejecting
  requests without the category payload (400).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(test): migrate categories API index/show tests to X-Api-Key

The pre-existing index and show tests in this file authenticated via
Doorkeeper bearer tokens. Per the project's API endpoint consistency
rule (CLAUDE.md, .cursor/rules/api-endpoint-consistency.mdc) Minitest
controller tests under test/controllers/api/v1/ must use ApiKey +
X-Api-Key auth. Drops the Doorkeeper application/access-token setup
and routes every request through the existing api_keys fixtures and
the api_headers helper, matching the create-action tests already in
this file (and the pattern used in sync/users/family_settings tests).

No behavioural change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(api): address second-round review on POST /api/v1/categories

- Add a 400 response block to the POST rswag example so the generated
  OpenAPI documents the missing-category-payload contract that
  BaseController#handle_bad_request already returns. Regenerate
  docs/api/openapi.yaml.
- Replace fixture-backed read_write_api_key / read_only_api_key
  helpers with explicit ApiKey.create! calls (matching the pattern in
  sync_controller_test, users_controller_test, and
  family_settings_controller_test). Setup now destroys active keys for
  the test user so the one-active-key-per-source validation does not
  collide with fixtures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(api): tighten 422 create-category cases

- Pass color and icon explicitly in the duplicate-name and
  third-level-nesting tests so each case is self-documenting about
  which validation it isolates (the model's color presence check is
  satisfied by the column default today, but reviewers — human and
  bot — flagged the implicit reliance).
- Assert the JSON error envelope (error key + present message) on every
  422 path so the response shape stays consistent and a regression in
  the rendered error body is caught uniformly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(api): tighten POST /api/v1/categories per review

- Drop the no-op `rescue ActionController::ParameterMissing; raise` and
  the broad `rescue => e` from the create action. The BaseController
  already has rescue_from ActionController::ParameterMissing → 400, and
  unexpected exceptions are best left to Rails' default 500 handling
  (which logs identically). Keeps the action focused on its happy path
  and the two real error branches.
- Stop accepting `lucide_icon` as a request key. The OpenAPI schema
  documents only `icon`; the dual permit was undocumented and pointless.
  `icon` is now the single canonical request key, mapped to
  `lucide_icon` on the model in category_params.
- Migrate the Minitest helpers to the project's documented API key
  pattern: ApiKey.generate_secure_key + api_key.plain_key in the
  X-Api-Key header (matching the rswag spec in this PR and the rule in
  .cursor/rules/api-endpoint-consistency.mdc), instead of hand-built
  display_key strings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Botched conflict merge

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-05-06 22:59:55 +02:00
ghost
9e369831ce feat(api): expose sync status (#1635)
* feat(api): expose sync status

* fix(api): harden sync status review paths

* fix(api): address sync status review

* fix(api): tighten sync status review fixes

* fix(api): address sync status review

* test(api): avoid secret-like sync fixture key

* test(api): reuse sync status fixture key

* fix(api): align sync route helpers

* fix(api): tighten sync status scoping

* fix(api): make sync status schema nullable-compliant
2026-05-06 22:02:21 +02:00
ghost
2d38cfb011 feat(api): expose budget state (#1640)
* feat(api): expose budget state

* fix(api): guard malformed budget ids

* fix(api): address budget state review

* fix(api): address budget state review

* fix(api): document budget id formats

* fix(api): align budget category docs auth

* fix(api): lighten budget category index payload

* fix(api): use shared pagination clamp

* fix(api): centralize budget filter handling
2026-05-06 20:50:46 +02:00
ghost
41339b0494 feat(api): expose balance history (#1641)
* feat(api): expose balance history

* fix(api): address balance history review

* fix(api): address balance history review

* fix(api): tighten balance history docs

* fix(exports): preserve balance chronology

* fix(api): guard nullable balance account type

* test(api): align balances api key helper

* fix(api): use shared pagination clamp

* test(export): set explicit balance flows factor
2026-05-05 19:09:36 +02:00
ghost
1ec8bd90b7 feat(api): expose import row diagnostics (#1644)
* feat(api): expose import row diagnostics

* fix(api): stabilize import row diagnostics

* fix(api): harden import row diagnostics

* fix(api): number Mint import diagnostics rows

* fix(api): enforce unique import row diagnostics

* fix(api): address import row diagnostics review
2026-05-05 01:12:48 +02:00
ghost
a48f264799 feat(api): expose securities and price history (#1642)
* feat(api): expose securities and prices

* fix(api): stabilize security price filters

* fix(api): cap security pagination limits

* fix(api): preserve security price decimal scale

* fix(api): validate securities boolean filters

* fix(api): reject blank securities boolean filters

* fix(api): trim security exchange filter

* fix(api): tighten security price filters

* fix(api): tighten security resource filters

* fix(api): tighten securities docs fixtures
2026-05-05 01:08:43 +02:00
ghost
05ef8bd9e7 feat(api): support idempotent valuation writes (#1637)
* feat(api): support idempotent valuation writes

* fix(api): clarify valuation upsert status

* docs(api): document nested valuation upserts

* docs(api): clarify valuation upsert semantics

* docs(api): clarify valuation upsert signaling
2026-05-04 18:51:48 +02:00
ghost
9cb3b8e05c feat(api): expose rule run history (#1646)
* feat(api): expose rule run history

* fix(api): address rule run review

* fix(api): complete rule run review

* test(api): cover unauthenticated rule run show

* test(api): align rule run api key helper

* Small Sonnet nit-pick

---------

Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-05-03 23:33:35 +02:00
ghost
e93b1f1fd7 feat(api): expose family settings (#1645)
* feat(api): expose family settings

* test(api): assert family settings moniker

* test(api): align family settings api key helper

* fix(api): tighten family settings schema
2026-05-03 23:10:46 +02:00
ghost
911aa34ba9 feat(auth): add WebAuthn MFA credentials (#1628)
* feat(auth): add WebAuthn MFA credentials

* fix(auth): harden WebAuthn MFA review paths

* fix(auth): polish WebAuthn error handling

* fix(auth): handle duplicate WebAuthn credential races

* fix(auth): permit WebAuthn credential params

* fix(auth): trim WebAuthn registration controller cleanup

* fix(auth): tighten WebAuthn MFA handling

* fix(auth): pin WebAuthn relying party config
2026-05-03 22:13:28 +02:00
ghost
50936000e7 feat(api): expose family exports (#1632)
* feat(api): expose family exports

* fix(api): harden family export review paths

* fix(api): tighten family export review paths

* fix(api): reject invalid family export params

* fix(api): address family export review

* fix(api): share uuid guard for exports
2026-05-03 11:29:29 +02:00
ghost
a8425a2488 feat(api): expose reset status polling (#1598)
* feat(api): expose reset status polling

* fix(api): hide reset enqueue exception details

* fix(api): use stable reset authorization message

* fix(api): narrow reset enqueue error handling

* fix(api): document reset enqueue failures

* docs(api): regenerate reset status OpenAPI

* fix(api): address reset polling review feedback
2026-05-02 22:56:42 +02:00
ghost
c4414c4fbb feat(api): expose import status details (#1599)
* feat(api): expose import status details

* fix(api): reuse import status validation counts

* fix(api): cache Sure import status reads

* fix(imports): invalidate cached Sure import blobs

* docs(api): split import status schemas

* fix(api): refine import status detail contract
2026-05-01 22:59:32 +02:00
ghost
da42423475 feat(api): accept Sure NDJSON imports (#1601)
* feat(api): accept Sure NDJSON imports

* fix(api): preserve uploaded Sure imports on publish errors

* fix(api): reset preserved Sure imports after enqueue failure

* fix(api): tighten Sure import upload handling

* test(api): align import API key fixtures

* docs(api): document import publish failure IDs
2026-05-01 22:56:18 +02:00
ghost
b710b55124 feat(api): add recurring transaction endpoints (#1600)
* feat(api): add recurring transaction endpoints

* fix(api): return validation errors for recurring writes

* fix(api): harden recurring transaction request handling

* fix(api): require writable recurring account access

* fix(api): default null recurring manual flag

* fix(api): tighten recurring transaction contracts

* test(api): align recurring transaction fixtures

* docs(api): regenerate recurring transaction OpenAPI
2026-05-01 21:21:34 +02:00
ghost
783309188f feat(api): expose rule export endpoints (#1602)
* feat(api): expose rule export endpoints

* fix(api): tighten rule export contracts

* fix(api): document balance sheet auth errors

* test(api): align rule API key fixtures

* Update docs/api/openapi.yaml

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>

* Quick win

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-01 19:47:06 +02:00
ghost
352c301e4b feat(api): expose valuation history index (#1596)
* feat(api): expose valuation history index

* fix(api): hide valuation exception details

* fix(api): reuse eager-loaded valuation entries

* fix(api): tighten valuation index contracts

* fix(api): scope valuation filter errors

* docs(api): nest valuation account filter format

* Fix merge conflict mistakes

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-05-01 19:09:56 +02:00
ghost
cc043b5caf feat(api): expose complete account export state (#1597)
* feat(api): expose complete account export state

* fix(api): handle malformed account identifiers

* fix(api): tighten account export contracts

* fix(api): correct account id OpenAPI format

* fix(api): tighten account docs auth contracts

* docs(api): document balance sheet auth errors

* docs(api): clarify account scope fixture
2026-05-01 15:22:28 +02:00
LPW
b457514c31 chore(pipelock): bump chart default to v2.2.0, add CI scan badge (#1494)
- Helm chart default pipelock.image.tag bumped from 2.0.0 to 2.2.0
  (three minor releases behind latest)
- README: pipelock CI scan status badge added to the existing badge row
- charts/sure/README.md, docs/hosting/pipelock.md, pipelock.example.yaml:
  refreshed feature notes to reference the upstream changelog rather than
  pinning to a single version
- compose.example.ai.yml: pin example comment bumped to :2.2.0
- Workflow pin (@v2) unchanged — floating major tag picks up 2.2.x
2026-04-18 09:32:23 +02:00
soky srm
90b1308866 Ipv6 support (#1437)
* Ipv6 support

* Proper fix for containers, dev and local

* Edits similar to non-AI compose file

---------

Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-04-13 13:44:37 +02:00
soky srm
0aca297e9c Add binance security provider for crypto (#1424)
* Binance as securities provider

* Disable twelve data crypto results

* Add logo support and new currency pairs

* FIX importer fallback

* Add price clamping and optiimize retrieval

* Review

* Update adding-a-securities-provider.md

* day gap miss fix

* New fixes

* Brandfetch doesn't support crypto. add new CDN

* Update _investment_performance.html.erb
2026-04-10 15:43:22 +02:00
Copilot
1527611239 Default production SSO provider source to YAML to avoid boot-time schema errors (#1278)
* Initial plan

* Default production SSO provider source to YAML

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>
Agent-Logs-Url: https://github.com/we-promise/sure/sessions/d3a36ca8-e936-4687-a466-9b4c93c19150

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>
2026-03-25 15:08:36 +01:00
LPW
1ddc427fd5 chore(helm): bump pipelock to v2.0.0 with trusted domains and redirect profiles (#1266)
* chore(helm): bump pipelock to v2.0.0 with trusted domains and redirect profiles

- Bump pipelock image tag from 1.5.0 to 2.0.0
- Add first-class Helm values for trustedDomains and mcpToolPolicy.redirectProfiles
- Update CI GitHub Action from @v1 to @v2
- Update compose example, config reference, and docs with v2.0 features

* Releasing this today in `alpha` form

---------

Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-03-24 09:30:54 +01:00
LPW
e43a8f295d Bump pipelock image from 0.3.2 to 1.5.0 (#1238)
* Bump pipelock image from 0.3.2 to 1.5.0

* Releasing via `alpha`

---------

Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-03-21 15:32:07 +01:00
Dream
6d22514c01 feat(vector-store): Implement pgvector adapter for self-hosted RAG (#1211)
* Add conditional migration for vector_store_chunks table

Creates the pgvector-backed chunks table when VECTOR_STORE_PROVIDER=pgvector.
Enables the vector extension, adds store_id/file_id indexes, and uses
vector(1024) column type for embeddings.

* Add VectorStore::Embeddable concern for text extraction and embedding

Shared concern providing extract_text (PDF via pdf-reader, plain-text as-is),
paragraph-boundary chunking (~2000 chars, ~200 overlap), and embed/embed_batch
via OpenAI-compatible /v1/embeddings endpoint using Faraday. Configurable via
EMBEDDING_MODEL, EMBEDDING_URI_BASE, with fallback to OPENAI_* env vars.

* Implement VectorStore::Pgvector adapter with raw SQL

Replaces the stub with a full implementation using
ActiveRecord::Base.connection with parameterized binds. Supports
create_store, delete_store, upload_file (extract+chunk+embed+insert),
remove_file, and cosine-similarity search via the <=> operator.

* Add registry test for pgvector adapter selection

* Configure pgvector in compose.example.ai.yml

Switch db image to pgvector/pgvector:pg16, add VECTOR_STORE_PROVIDER,
EMBEDDING_MODEL, and EMBEDDING_DIMENSIONS env vars, and include
nomic-embed-text in Ollama's pre-loaded models.

* Update pgvector docs from scaffolded to ready

Document env vars, embedding model setup, pgvector Docker image
requirement, and Ollama pull instructions.

* Address PR review feedback

- Migration: remove env guard, use pgvector_available? check so it runs
  on plain Postgres (CI) but creates the table on pgvector-capable servers.
  Add NOT NULL constraints on content/embedding/metadata, unique index on
  (store_id, file_id, chunk_index).
- Pgvector adapter: wrap chunk inserts in a DB transaction to prevent
  partial file writes. Override supported_extensions to match formats
  that extract_text can actually parse.
- Embeddable: add hard_split fallback for paragraphs exceeding CHUNK_SIZE
  to avoid overflowing embedding model token limits.

* Bump schema version to include vector_store_chunks migration

CI uses db:schema:load which checks the version — without this bump,
the migration is detected as pending and tests fail to start.

* Update 20260316120000_create_vector_store_chunks.rb

---------

Co-authored-by: sokiee <sokysrm@gmail.com>
2026-03-20 17:01:31 +01:00
Andrei Onel
a0b1029ba9 Documentation for review AI Assistant features, MCP and API additions (#1168)
* Create MCP server endpoint documentation

* Add Assistant Architecture section to AI documentation

* Add Users API documentation for account reset and delete endpoints

* Document Pipelock CI security scanning in contributing guide

* fix: correct scope and error codes in Users API documentation

* Exclude `docs/hosting/ai.md` from Pipelock scan

---------

Co-authored-by: askmanu[bot] <192355599+askmanu[bot]@users.noreply.github.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-03-16 18:24:28 +01:00
Juan José Mata
cade5b22f7 Document admin-only reset auth in OpenAPI docs (#1198)
* Document admin-only reset auth in OpenAPI docs

The DELETE /api/v1/users/reset endpoint now requires admin role
(ensure_admin). Update the rswag spec to:
- Set default user role to admin so the 200 test passes
- Add a 403 response case for non-admin users with read_write scope
- Clarify the description notes admin requirement
- Add SuccessMessage schema and users paths to openapi.yaml

https://claude.ai/code/session_01Tj8ToLRmVg5HLmHwq9KKDY

* Consolidate duplicate 403 responses for reset endpoint

OpenAPI keys responses by status code, so two 403 blocks caused the
first (insufficient scope) to be silently overwritten by the second
(non-admin). Merge into a single 403 whose description covers both
causes: requires read_write scope and admin role. The test exercises
the read-only key path which hits 403 via scope check.

https://claude.ai/code/session_01Tj8ToLRmVg5HLmHwq9KKDY

* Em-dash out of messages.

* Fix tests

* Fix tests

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-15 00:23:38 +01:00
LPW
ca8f04040f Expand AI docs: external assistant, MCP, architecture, troubleshooting (#1115)
* Expand AI docs: architecture, MCP, external assistant setup, troubleshooting

- Add architecture overview explaining two independent AI pipelines
  (chat assistant vs auto-categorization)
- Document MCP callback endpoint (JSON-RPC 2.0, auth, available tools)
- Add OpenClaw gateway configuration example
- Add Kubernetes network policy guidance (targetPort vs servicePort)
- Add Pipelock notes (mcpToolPolicy, NO_PROXY behavior)
- Add troubleshooting for "Failed to generate response" with external assistant
- Fix stale function list (4 tools -> 7)
- Fix incorrect env-vs-UI precedence statement
- Fix em-dashes in existing content

* Fix troubleshooting curl to use pod env vars

Use sh -c so $EXTERNAL_ASSISTANT_TOKEN and $EXTERNAL_ASSISTANT_URL
expand inside the pod, not on the local shell.
2026-03-04 11:26:43 +01:00
LPW
a53a131c46 Add Pipelock operational templates, docs, and config hardening (#1102)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.

* Pipelock Helm hardening + docs for external assistant and pipelock

Helm templates:
- ServiceMonitor for Prometheus scraping on /metrics (proxy port)
- Ingress template for MCP reverse proxy (external AI agent access)
- PodDisruptionBudget with minAvailable/maxUnavailable mutual exclusion
- topologySpreadConstraints on Deployment
- Structured logging config (format, output, include_allowed/blocked)
- extraConfig escape hatch for additional pipelock.yaml sections
- requireForExternalAssistant guard (fails when assistant enabled without pipelock)
- Component label on Service metadata for ServiceMonitor targeting
- NOTES.txt pipelock section with health, access, security, metrics info
- Bump pipelock image tag 0.3.1 -> 0.3.2
- Fix: rename _asserts.tpl -> asserts.tpl (Helm skipped _ prefixed file)

Documentation:
- Helm chart README: full Pipelock section
- docs/hosting/pipelock.md: dedicated hosting guide (Docker + Kubernetes)
- docs/hosting/docker.md: AI features section (external assistant, pipelock)
- .env.example: external assistant and MCP env vars

Infra:
- Chart.lock pinning dependency versions
- .gitignore for vendored subchart tarballs

* Fix bot comments: quote ingress host, fix sidecar wording, add code block lang

* Fail fast when pipelock ingress enabled with empty hosts

* Fail fast when pipelock ingress host has empty paths

* Messed up the conflict merge

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-03-03 16:32:35 +01:00
LPW
84bfe5b7ab Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
2026-03-03 15:47:51 +01:00
Juan José Mata
bf0be85859 Expose ui_layout and ai_enabled to mobile clients and add enable_ai endpoint (#983)
* Wire ui layout and AI flags into mobile auth

Include ui_layout and ai_enabled in mobile login/signup/SSO payloads,
add an authenticated endpoint to enable AI from Flutter, and gate
mobile navigation based on intro layout and AI consent flow.

* Linter

* Ensure write scope on enable_ai

* Make sure AI is available before enabling it

* Test improvements

* PR comment

* Fix review issues: test assertion bug, missing coverage, and Dart defaults (#985)

- Fix login test to use ai_enabled? (method) instead of ai_enabled (column)
  to match what mobile_user_payload actually serializes
- Add test for enable_ai when ai_available? returns false (403 path)
- Default aiEnabled to false when user is null in AuthProvider to avoid
  showing AI as available before authentication completes
- Remove extra blank lines in auth_provider.dart and auth_service.dart

https://claude.ai/code/session_01LEYYmtsDBoqizyihFtkye4

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-14 00:39:03 +01:00
Juan José Mata
34afc1f597 Document merchants API endpoints (#980)
Add rswag request specs for merchants index/show and define a MerchantDetail schema used by the docs. Update the generated OpenAPI document with merchants paths and schema.
2026-02-13 19:40:23 +01:00
Juan José Mata
4e4ca916a1 Update backend table with status and requirements
Clarify status of non-OpenAI vector store

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
2026-02-11 15:59:12 +01:00