mirror of
https://github.com/we-promise/sure.git
synced 2026-07-12 21:05:20 +00:00
e75f2a0c78b0cd1199636e0979ccf3bd14fd1ea6
27 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Guillem Arias Fauste
|
c375b8bf5c |
feat(ai): self-host settings UI for Anthropic provider (5/5) (#1987)
* feat(ai): add Anthropic provider with chat parity (1/5)
Introduces Provider::Anthropic alongside Provider::Openai, implementing
the LlmConcept chat_response contract over the official anthropic Ruby
SDK. Batch ops, PDF, and RAG land in follow-up PRs.
- Provider::Anthropic uses Messages API for sync and streaming responses
- ChatConfig builds requests with ephemeral prompt-cache markers on the
system prompt and the last tool definition
- MessageFormatter reconstructs multi-turn history (text + tool_use +
tool_result blocks) from raw Message records, including the paired
user-role tool_result turn Anthropic requires after every tool_use
- ChatParser maps Anthropic Message into the shared ChatResponse Data
- Registry, Setting, User, Chat default model wired for ANTHROPIC_*
envs and Setting.anthropic_*; LLM_PROVIDER selects between providers
- Responder forwards raw conversation_history (Array<Message>) so
providers without hosted conversation state can rebuild context
- OpenAI provider accepts and ignores the new kwarg (no behavior change)
Tests cover provider init, model gating, MessageFormatter for all turn
shapes, ChatConfig request building (max_tokens, system cache, tool
conversion), ChatParser for text / tool_use / mixed blocks, Registry
discovery, and mocked chat_response success / error / function_request
paths. Live VCR cassettes recorded in a follow-up with a real key.
Stacked PRs: 2/5 batch ops + cost ledger, 3/5 PDF, 4/5 pgvector RAG,
5/5 settings UI + disclosure.
* fix(ai): address PR review on Anthropic provider foundation
Surface fixes raised by Codex + CodeRabbit on PR 1/5:
- Provider::Anthropic#chat_response now accepts (and ignores) a
`messages:` kwarg. Assistant::Responder passes both `messages:`
(OpenAI-shape) and `conversation_history:` (raw Message records) for
cross-provider parity, so the previous signature raised
ArgumentError on the first chat turn through the Anthropic provider.
- Provider::Anthropic#supports_model? bypasses the `claude` prefix
gate when a custom base_url is configured, mirroring the OpenAI
provider. Bedrock-shaped IDs like
`anthropic.claude-sonnet-4-5-20250929-v1:0` and
`claude-opus-4@20250514` are otherwise rejected by
Assistant::Provided#get_model_provider and the chat dies.
- Setting.anthropic_access_token is now in
EncryptedSettingFields::ENCRYPTED_FIELDS so the Anthropic API key
is encrypted at rest like every other provider secret. Previously
plaintext while siblings (openai_access_token, twelve_data_api_key,
external_assistant_token) were ciphertext.
- Chat.default_model falls back to whichever provider is actually
configured. Previously, with LLM_PROVIDER=anthropic but no
Anthropic credentials, the default model resolved to a Claude ID
that no registered provider supported, so chats failed even when
OpenAI was fully configured. Adds Provider::{Anthropic,Openai}#configured?
class methods for the readable callsite.
- Provider::Anthropic.effective_model uses
`ENV["ANTHROPIC_MODEL"].presence || Setting.anthropic_model` so the
Setting lookup is only performed when the env var is absent — the
previous `ENV.fetch(KEY, default)` evaluated the default arg
eagerly on every call.
- Provider::Anthropic::ChatConfig#anthropic_input_schema strips both
`:strict` and `"strict"` keys so JSON-decoded schemas with string
keys cannot leak the OpenAI-only flag through to Anthropic.
Test coverage added: supports_model? bypass on custom endpoints,
chat_response messages: kwarg compatibility, default_model fallback
in the three credential combinations, configured? against ENV +
Setting, strict-flag stripping for both key types, and a
`Setting.expects(:anthropic_model).never` assertion proving the
ENV-precedence test now exercises the lazy path.
All 4365 tests pass (1 pre-existing libvips env error unrelated).
* test(chat): make default_model tests resilient to ENV model overrides
CodeRabbit flagged on PR review: the new default_model tests asserted
against Provider::*::DEFAULT_MODEL, but Chat.default_model actually
returns Provider::*.effective_model.presence (which reads
OPENAI_MODEL / ANTHROPIC_MODEL from the environment). With either env
var set, the tests would fail intermittently even though routing was
correct.
- New default_model tests now assert against the provider's
effective_model directly, so they verify the routing decision
(which provider's value wins) without coupling to the constant.
- Pre-existing "creates with default model" assertions had the same
brittleness; switch them to compare against Chat.default_model so
the chosen model is whatever the env / Setting cascade resolves to.
Verified by running `ANTHROPIC_MODEL=claude-haiku-4-5 OPENAI_MODEL=gpt-4o
bin/rails test test/models/chat_test.rb` — 16 runs, 0 failures
(previously 2 pre-existing failures + 0 from the new tests).
* fix(ai): address local review on Anthropic foundation
- Provider::Anthropic#supports_pdf_processing? bypasses prefix gate for
custom endpoints, mirroring supports_model?
- Provider::Anthropic#initialize raises Error when custom_endpoint? AND
model.blank?, parity with Provider::Openai
- stream_chat_response captures partial usage on mid-stream errors and
records it via the new on_partial callback so chat_response can skip
the duplicate error row in the outer rescue
- safe_accumulated_message swallows the secondary failure when the SDK
cannot reconstruct a snapshot
- langfuse_client memoizes properly (||= instead of =) so repeated calls
don't churn Langfuse instances
- MessageFormatter sorts tool_calls by created_at then id so the
message array is deterministic across replays; skips tool_calls
missing both provider_call_id and provider_id rather than sending
`id: nil` and getting rejected by Anthropic
- Setting.anthropic_access_token default falls back through
ENV["ANTHROPIC_API_KEY"].presence (was missing .presence, so an
empty-string env value bled through)
- User#openai_configured? / #anthropic_configured? delegate to the
Provider::* class methods — single source of truth
- Assistant::Responder renames the OpenAI-shape history builder
conversation_history → openai_messages_payload so the kwarg name
matches the local method name (messages: openai_messages_payload,
conversation_history: chat_message_records)
- Assistant::Builtin stale-history comment updated to reference both
builders
Adds a streaming chat_response test using ad-hoc subclasses of the
SDK event types so the case/when dispatch matches via is_a? without
stubbing class-level === behavior.
* test(ai): add Anthropic tool_use round-trip + multi-tool turn coverage
Addresses @jjmata's "worth confirming" note on PR #1983: tool-use turns
from prior assistant messages must round-trip correctly when retrieved
from the database.
- New `ChatParser → ToolCall::Function → MessageFormatter` test walks
the full path: Anthropic response with a tool_use block →
ChatFunctionRequest → ToolCall::Function.from_function_request →
persisted on the AssistantMessage → MessageFormatter rebuild on the
next turn. Asserts the original `tool_use.id` is preserved end-to-end
as both `tool_use.id` and the paired `tool_result.tool_use_id`, and
that the original `input` hash and serialized result content survive.
- New multi-tool assistant turn test confirms two tool_use blocks on a
single assistant message render as two tool_use blocks followed by
two paired tool_result blocks in a single user-role follow-up,
matching Anthropic's required alternation.
Both tests exercise the existing PR1 code without behavior changes.
* test(ai): require "ostruct" explicitly in Anthropic provider tests
OpenStruct is moving out of Ruby's default load path (warning in 3.4+,
removed in 3.5+). Tests work today because ActiveSupport transitively
loads it, but that's incidental. Match the existing convention in
test/controllers/settings/hostings_controller_test.rb which explicitly
requires ostruct for the same reason.
* fix(ai): sanitize Langfuse warn logs, normalize tool_use.input, dedup history fetch
Addresses three open CodeRabbit findings on PR #1983.
- Provider::Anthropic Langfuse rescue branches no longer include
`e.full_message` in `Rails.logger.warn`. `full_message` bundles the
backtrace + cause chain and on some SDK error types includes the
serialized request/response payload (prompt, model output). Logs
now report `#{e.class}: #{e.message}` only. Three sites:
create_langfuse_trace, log_langfuse_generation, upsert_langfuse_trace.
Note: Provider::Openai has the same pattern (copy-pasted source) —
harmonization deferred to a follow-up cleanup PR; this commit fixes
only the Anthropic provider to keep PR scope tight.
- MessageFormatter#parse_arguments now coerces any non-Hash parsed
result to `{}`. Anthropic's Messages API requires `tool_use.input`
to be a JSON object (map); a stored ToolCall::Function record whose
arguments parse to a scalar, bool, or array (corrupt row, legacy
data, cross-provider bleed) would otherwise produce a payload the
API rejects. Normal flow stores Hash arguments end-to-end so the
fix is defensive — adds 2 tests covering scalar/array JSON strings
and non-String non-Hash inputs.
- Assistant::Responder dedups the chat-history fetch. The previous
layout fired two near-identical `chat.messages.where(...).includes(
:tool_calls).ordered` queries per LLM turn (one for the OpenAI-shape
payload, one for the raw-records kwarg). A new memoized
`complete_chat_messages` fetches once; `chat_message_records` filters
out the current message via `Array#reject`, `openai_messages_payload`
iterates the cached array unchanged. One SQL query per turn instead
of two. Memoization scope = single Responder instance (per LLM call),
so cache invalidation is not a concern.
All 4370 tests pass (1 pre-existing libvips env error unrelated).
Rubocop + brakeman clean.
* fix(ci): replace sk-ant- prefixed test placeholders
Pipelock secret scanner pattern-matches `sk-ant-*` as a real Anthropic
API key and fails the PR security-scan check. Test stubs and
ClimateControl env values used `sk-ant-test`, `sk-ant-from-setting`,
`sk-ant-x`, `sk-ant-y` as obvious placeholders, but the scanner does
not care about value entropy.
Switched to `fake-anthropic-key-*` / `fake-token-*` strings so the
scanner stops flagging them. No production code touched, no behavior
change — Provider::Anthropic still accepts any non-blank token.
* feat(ai): add Anthropic batch ops + LLM cost ledger (2/5)
Implements auto_categorize, auto_detect_merchants, and
enhance_provider_merchants on Provider::Anthropic via forced tool calls,
plus the cost-ledger plumbing they need.
- Provider::Anthropic::AutoCategorizer, AutoMerchantDetector,
ProviderMerchantEnhancer each define a single output tool whose
input_schema mirrors the desired output, then force the model to call
it via tool_choice: { type: "tool", name: ..., disable_parallel_tool_use: true }.
Anthropic guarantees the tool_use.input matches the schema, so there
is no JSON parsing fragility, no <think> tag stripping, and no
json_object/json_schema fallback ladders.
- Concerns::UsageRecorder mirrors the OpenAI sibling but persists
cache_creation_input_tokens / cache_read_input_tokens to dedicated
columns instead of metadata.
- Migration adds cache_creation_tokens, cache_read_tokens (nullable
integers) to llm_usages. OpenAI rows leave them null.
- LlmUsage::PRICING gains Claude 4.x rows (opus-4-7 $15/$75, sonnet-4-6
$3/$15, haiku-4-5 $1/$5 per MTok). infer_provider returns "anthropic"
for claude-* via the existing exact/prefix lookup.
- Provider::Anthropic#chat_response now persists cache columns directly
rather than stashing them in metadata.
- 25-transaction batch cap mirrors the OpenAI provider so the cost
ledger sees the same shape regardless of which provider ran a batch.
Tests cover the forced-tool-call path, null/None normalization,
case-insensitive merchant matching, the missing-tool_use error path,
and Anthropic-specific pricing + provider inference on LlmUsage.
Stacked on #1983 (PR 1/5). 3/5 PDF + vision next.
* fix(ai): attribute Bedrock model IDs to anthropic + clean nil enum
- LlmUsage.infer_provider now returns "anthropic" for Bedrock /
Vertex shaped IDs (anthropic.* and anthropic/*), so cost-ledger
filtering by provider stays correct even when no per-MTok rate is
stored. Previously these IDs fell through to the "openai" default.
- AutoCategorizer drops the redundant nil sentinel from the
category_name enum — the union type [string, null] already permits
null, and some JSON Schema validators reject nil literals inside
enum arrays.
* test(ai): require "ostruct" in Anthropic batch op tests
Same rationale as the PR1 ostruct fix — explicit require so the tests
don't depend on ActiveSupport's transitive load when Ruby 3.5+ removes
OpenStruct from the default load path.
* feat(ai): Anthropic native PDF processing (3/5)
Implements process_pdf and extract_bank_statement on Provider::Anthropic
using the native `document` content block — no rasterization, no text
pre-extraction.
- Provider::Anthropic::PdfProcessor classifies the document, summarizes
it, and extracts statement metadata via a forced report_document_analysis
tool whose input_schema mirrors the existing Provider::Openai output
(document_type from Import::DOCUMENT_TYPES, summary, extracted_data).
- Provider::Anthropic::BankStatementExtractor returns the same
{ transactions, period, account_holder, account_number, bank_name,
opening_balance, closing_balance } shape via report_bank_statement so
downstream pdf_import code is provider-agnostic.
- Both attach the PDF as
{ type: "document", source: { type: "base64", media_type: "application/pdf", data: <b64> } }
— Claude 3.5+ / 4.x accept this natively (up to 32MB / 100 pages).
No pdf-reader, no pdftoppm, no chunking for typical statements.
- supports_pdf_processing? (introduced in PR 1) already returns true for
claude-* models, gating process_pdf with a clear error otherwise.
- Cost ledger rows are persisted via the shared UsageRecorder concern,
including cache_creation/cache_read tokens.
Tests verify the document block shape, tool_choice forcing, normalized
document_type for unknown classifications, transaction normalization
(date / amount / reference → notes), and the missing-tool_use error
path. Blank pdf_content raises before any client call.
Stacked on #1984 (PR 2/5). 4/5 pgvector RAG next.
* fix(ai): guard PDF size + surface bank-statement truncation
- PdfProcessor and BankStatementExtractor raise upfront when
pdf_content.bytesize exceeds MAX_PDF_BYTES (32 MB, matching
Anthropic's hard limit). Previously a 100 MB PDF would be
base64-encoded (~133 MB) and packed into the JSON body before
the API rejected it — peak heap ~270 MB per Sidekiq worker.
- BankStatementExtractor inspects response.stop_reason; when the
model hit max_tokens it logs a warning and flags result[:truncated]
so downstream callers know the transaction list may be incomplete.
- ISO date pattern added to statement_period_start/end schema in
PdfProcessor so the model can't return "March 2026" — Anthropic
enforces the regex via the tool's input_schema.
Tests cover the size guard (raises before any client.messages call),
truncated-result flagging, and the warning log path.
* test(ai): require "ostruct" in Anthropic PDF tests
Match the explicit ostruct require added in PR1/PR2 — same Ruby 3.5+
load-path reason.
* feat(ai): default Anthropic installs to pgvector RAG (4/5)
The provider-agnostic vector store stack (VectorStore::Pgvector + the
Embeddable concern) already shipped to main. This PR closes the
Anthropic loop:
- VectorStore::Registry.adapter_name now returns :pgvector when
Setting.llm_provider == "anthropic" and no explicit
VECTOR_STORE_PROVIDER override is set. Anthropic has no hosted vector
store, so falling back to the local pgvector adapter is the only
correct default. Explicit VECTOR_STORE_PROVIDER still wins.
- SearchFamilyFiles surfaces a longer message when no adapter is wired
up — calling out pgvector + EMBEDDING_URI_BASE as the supported
Anthropic-only path so the user is not stuck with an "OpenAI required"
hint that is no longer accurate.
The Embeddable concern already pulls embeddings from
EMBEDDING_URI_BASE / EMBEDDING_ACCESS_TOKEN (with OpenAI as fallback),
so Anthropic installs point this at Voyage AI, a local Ollama instance,
or OpenAI embeddings — independent of the chat provider.
Tests cover the new default routing, the existing OpenAI default
staying intact, and explicit VECTOR_STORE_PROVIDER overriding the
Anthropic default.
Stacked on #1985 (PR 3/5). 5/5 settings UI + retention disclosure next.
* feat(ai): self-host settings UI for Anthropic provider (5/5)
Adds the Anthropic panel and the install-wide LLM provider selector to
the self-hosting settings page, plus a shared data-retention
disclosure that covers both OpenAI and Anthropic.
- New _llm_provider_selector partial: select for Setting.llm_provider
(openai | anthropic), respects the LLM_PROVIDER env var (disables the
control + shows the "configured through environment variables" hint
when set, mirroring the existing OpenAI panel behaviour), and renders
a compact data-handling block with one-line retention statements for
each provider.
- New _anthropic_settings partial mirrors _openai_settings exactly:
password-field for the API key with **** redaction, optional
base_url (for AWS Bedrock / GCP Vertex), optional default model. All
three fields disable when their ENV var is set.
- show.html.erb renders provider selector + OpenAI panel + Anthropic
panel under the same "General" section so users can configure either
(or both) without switching pages.
- Settings::HostingsController#update now permits and persists
anthropic_access_token (ignoring the **** placeholder, same pattern
as OpenAI), anthropic_base_url, anthropic_model, and llm_provider
(validated against %w[openai anthropic]). On Setting::ValidationError
the rescue branch preserves anthropic_base_url / anthropic_model
input so the form re-renders with the user's typed values intact —
parity with the issue #1824 fix for OpenAI.
- Locale keys added under settings.hostings.{llm_provider_selector,
anthropic_settings}.
Tests cover token update + placeholder redaction, base_url + model
update, llm_provider switch to anthropic, and rejection of unknown
provider values. The existing GET render test still passes, exercising
all three new partials.
Closes the 5/5 Anthropic series stacked on #1986.
* fix(ai): valid Tailwind token + base_url URL validation
- Data-handling block in _llm_provider_selector swaps the invalid
bg-surface-secondary token for bg-container-inset, matching the
inset-card pattern used elsewhere in sure-design-system/components.css.
bg-surface-secondary is not defined anywhere in the design system —
Tailwind treated it as a no-op, so the block rendered with no
background contrast.
- Settings::HostingsController validates anthropic_base_url as a
URI::HTTP (catches https too) and raises Setting::ValidationError
with a localized message when the input is not parseable.
Previously any string was persisted, surfacing as an opaque
connection error at request time instead of an immediate UX failure.
- Blank base_url now clears the setting (was already the case but
exercised explicitly in tests now).
* fix(ci): replace sk-ant- prefixed token in hostings controller test
Same pipelock secret-scan trigger as PR1 fix on registry/anthropic
tests. The sk-ant-* prefix is matched verbatim by the scanner
regardless of value entropy.
* fix(ai): provision pgvector table when it is the default store
#1986 makes pgvector the default vector store for Anthropic installs, but
CreateVectorStoreChunks only ran when VECTOR_STORE_PROVIDER=pgvector was set
explicitly — so a fresh Anthropic-only install migrated without the
vector_store_chunks table and failed on uploads/searches.
Add VectorStore::Registry.pgvector_effective? as the single source of truth
for "is pgvector active?" (explicit env OR the Anthropic default), and a new
idempotent migration that enables the extension + creates the table whenever
pgvector is effective and the table is missing — covering fresh and
already-migrated installs without drift. Addresses Codex P1.
* fix(ai): provision pgvector table for Anthropic-default installs
Migration gated on raw VECTOR_STORE_PROVIDER==pgvector, so an
Anthropic-default install (which selects pgvector implicitly via
Setting.llm_provider without setting VECTOR_STORE_PROVIDER) skipped
table creation and failed later on a missing vector_store_chunks
relation. Route through VectorStore::Registry.pgvector_effective? —
the single source of truth already shared by the adapter selection.
Addresses Codex P1 review finding.
* fix(ai): provision pgvector chunks table on schema-load installs
The ensure-migration only helps db:migrate upgraders. Fresh installs go
through bin/docker-entrypoint's db:prepare, which loads schema.rb (the
conditional table can't be dumped there — it needs the vector extension)
and marks every migration applied without running it. An Anthropic-only
fresh install therefore selected the pgvector adapter but had no table,
failing with raw PG errors on first upload or search.
Two layers close it:
- VectorStore::Pgvector#ensure_schema! provisions the table idempotently
on first use (mirrors CreateVectorStoreChunks; memoized; failures wrap
in VectorStore::Error, which with_response turns into a clean failed
response).
- VectorStore::Registry#build_pgvector now gates on
VectorStore::Pgvector.available? (table exists, or extension present),
so installs whose Postgres lacks pgvector entirely degrade to the
assistant's provider_not_configured message instead of raising
mid-chat.
Also resolves the schema.rb version conflict against main (keep the
branch's 2026_06_01_120000, on top of main's current tables).
* fix(ai): address review nitpicks on pgvector provisioning
- Registry: update the adapter doc comment to mention the
Anthropic-to-pgvector default alongside the openai fallback.
- ensure_schema!: guard the DDL with if_not_exists instead of a Mutex.
Adapter instances are built per call and never shared across threads,
so the realistic race is two processes (web + Sidekiq) provisioning
concurrently; IF NOT EXISTS makes the loser a no-op where a Mutex
would only serialize threads inside one process.
* fix(ai): address review on Anthropic settings UI
- Require an Anthropic model when a custom base URL is saved, mirroring the
OpenAI branch. Auto-submit-on-blur could persist a base URL with no model,
making Provider::Anthropic raise "Model is required..." on every LLM call.
- Narrow the LLM provider selector copy: only chat honors Setting.llm_provider;
categorization, merchant detection and PDF processing still always use OpenAI.
Stop advertising provider switching for those flows until they are wired.
- Reset global Setting.* in test teardown to prevent state leakage, and add a
test covering the new base-URL-requires-model validation.
* feat(ds): conditional LLM provider settings + merged copy
The self-hosting AI section showed both providers' credential blocks at once
and duplicated near-identical copy. Tidy it:
- Replace the provider <select> with a DS::SegmentedControl driving a new
provider-settings Stimulus controller: only the active provider's panel is
shown; switching reveals the other instantly and persists Setting.llm_provider.
- Merge the two byte-identical data-retention lines into one provider-neutral
Data handling note.
- Scope the token-budget copy to OpenAI-compatible calls (read only by
Provider::Openai) and add an inline 'add a key to activate' hint when the
active provider is unconfigured.
UI-only; no provider behavior change.
* feat(ds): responsive LLM provider picker (tabs >=sm, select on mobile)
The segmented tabs overflow a phone viewport once there are 3+ providers
(measured: 4 labels want ~409px in a 319px column at 390px wide). Below sm,
fall back to a native <select> -- which doubles as the submitted field -- while
keeping the segmented tabs at sm and up.
Both controls bind to the same provider-settings Stimulus controller (the
select reads its value, the tabs read data-provider), so adding a 3rd/4th
provider scales on mobile with no layout math.
* fix(hostings): sanitize llm provider selector
* test(hostings): avoid brittle provider hint assertion
---------
Co-authored-by: sure-admin <sure-admin@splashblot.com>
|
||
|
dripsmvcp
|
8f5454ad29 |
fix(settings): preserve OpenAI form input on validation failure (#1862)
* fix(settings): preserve OpenAI form input on validation failure Fixes #1824. The OpenAI settings form auto-submits on blur, so typing the URI base before the model triggers cross-field validation. The rescue re-renders the page with values read from Setting.openai_*, which is still blank because the failed save was rejected — so the user's input disappears and they see 'OpenAI model is required' with no value to fix. Stash the submitted uri_base and model on rescue and prefer them over the saved Setting when rendering, so the user can finish typing the missing field and re-submit. * test(settings): cover openai_model preservation on validation fail (#1862) jjmata asked for symmetric coverage of the model field. Add a test where the user changes the URI base and clears the model in the same submit: the cross-field validation fails and the re-rendered model input must reflect the submitted (cleared) value rather than reverting to the saved model. Complements the existing uri_base preservation test. |
||
|
Juan José Mata
|
7b2b1dd367 |
Rebase PR #784 and fix OpenAI model/chat regressions (#1384)
* Wire conversation history through OpenAI responses API * Fix RuboCop hash brace spacing in assistant tests * Pipelock ignores * Batch fixes --------- Co-authored-by: sokiee <sokysrm@gmail.com> |
||
|
soky srm
|
7908f7d8a4 |
Expand financial providers (#1407)
* Initial implementation * Tiingo fixes * Adds 2 providers, remove 2 * Add extra checks * FIX a big hotwire race condition // Fix hotwire_combobox race condition: when typing quickly, a slow response for // an early query (e.g. "A") can overwrite the correct results for the final query // (e.g. "AAPL"). We abort the previous in-flight request whenever a new one fires, // so stale Turbo Stream responses never reach the DOM. * pipelock * Update price_test.rb * Reviews * i8n * fixes * fixes * Update tiingo.rb * fixes * Improvements * Big revamp * optimisations * Update 20260408151837_add_offline_reason_to_securities.rb * Add missing tests, fixes * small rank tests * FIX tests * Update show.html.erb * Update resolver.rb * Update usd_converter.rb * Update holdings_controller.rb * Update holdings_controller.rb * Update holdings_controller.rb * Update holdings_controller.rb * Update holdings_controller.rb * Update _yahoo_finance_settings.html.erb |
||
|
Juan José Mata
|
7ae9077935 |
Add default family selection for invite-only onboarding mode (#1174)
* Add default family selection for invite-only onboarding mode When onboarding is set to invite-only, admins can now choose a default family that new users without an invitation are automatically placed into as members, instead of creating a new family for each signup. https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx * Restrict invite codes and onboarding settings to super_admin only The Invite Codes section on /settings/hosting was visible to any authenticated user via the show action, leaking all family names/IDs through the default-family dropdown. This tightens access: - Hide the entire Invite Codes section in the view behind super_admin? - Add before_action :ensure_super_admin to InviteCodesController for all actions (index, create, destroy), replacing the inline admin? check - Add ensure_super_admin_for_onboarding filter on hostings#update that blocks non-super_admin users from changing onboarding_state or invite_only_default_family_id https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx * Fix tests for super_admin-only invite codes and onboarding settings - Hostings controller test: sign in as sure_support_staff (super_admin) for the onboarding_state update test, since ensure_super_admin_for_onboarding now requires super_admin role - Invite codes tests: use super_admin fixture for the success case and verify that a regular admin gets redirected instead of raising StandardError https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx * Fix system test to use super_admin for self-hosting settings The invite codes section is now only visible to super_admin users, so the system test needs to sign in as sure_support_staff to find the onboarding_state select element. https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx * Skip invite code requirement when a default family is configured When onboarding is invite-only but a default family is set, the claim_invite_code before_action was blocking registration before the create action could assign the user to the default family. Now invite_code_required? returns false when invite_only_default_family_id is present, allowing codeless signups to land in the configured default family. https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Juan José Mata
|
c09362b880 |
Check for pending invitations before creating new Family during SSO log in/sign up (#1171)
* Check for pending invitations before creating new Family during SSO account creation When a user signs in via Google SSO and doesn't have an account yet, the system now checks for pending invitations before creating a new Family. If an invitation exists, the user joins the invited family instead. - OidcAccountsController: check Invitation.pending in link/create_user - API AuthController: check pending invitations in sso_create_account - SessionsController: pass has_pending_invitation to mobile SSO callback - Web view: show "Accept Invitation" button when invitation exists - Flutter: show "Accept Invitation" tab/button when invitation pending https://claude.ai/code/session_019Tr6edJa496V1ErGmsbqFU * Fix external assistant tests: clear Settings cache to prevent test pollution The tests relied solely on with_env_overrides to clear configuration, but rails-settings-cached may retain stale Setting values across tests when the cache isn't explicitly invalidated. Ensure both ENV vars AND Setting values are cleared with Setting.clear_cache before assertions. https://claude.ai/code/session_019Tr6edJa496V1ErGmsbqFU --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Serge L
|
a92fd3b3e8 |
feat: Enhance holding detail drawer with live price sync and enriched overview (#1086)
* Feat: Implement manual sync prices functionality and enhance holdings display * Feat: Enhance sync prices functionality with error handling and update UI components * Feat: Update sync prices error handling and enhance Spanish locale messages * Fix: Address CodeRabbit review feedback - Set fallback @provider_error when prices_updated == 0 so turbo stream never fails silently without a visible error message - Move attr_reader :provider_error to class header in Price::Importer for conventional placement alongside other attribute declarations - Precompute @last_price_updated in controller (show + sync_prices) instead of running a DB query directly inside ERB templates Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix: Replace bare rescue with explicit exception handling in turbo stream view Bare `rescue` silently swallows all exceptions, making debugging impossible. Match the pattern already used in show.html.erb: rescue ActiveRecord::RecordInvalid explicitly, then catch StandardError with logging (message + backtrace) before falling back to the unknown label. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix: Update test assertion to expect actual provider error message The stub returns "Yahoo Finance rate limit exceeded" as the provider error. After the @provider_error fallback fix, the controller now correctly surfaces the real provider error when present (using .presence || fallback), so the flash[:alert] is the actual error string, not the generic fallback. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix: Assert scoped security_ids in sync_prices materializer test Replace loose stub with constructor expectation to verify that Balance::Materializer is instantiated with the single-security scope. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix: Assert holding remap in remap_security test Add assertion that @holding.security_id is updated to the target security after remap, covering the core command outcome. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix: CI test failure - Update disconnect external assistant test to use env overrides --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
LPW
|
84bfe5b7ab |
Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose - Add ConfigMap template rendering DLP, response scanning, MCP input/tool scanning, and forward proxy settings from values - Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment - Add checksum/config annotation for automatic pod restart on config change - Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip in MCP-only mode) - Use hasKey for all boolean values to prevent Helm default swallowing false - Single source of truth for ports (forwardProxy.port/mcpProxy.port) - Pipelock-specific imagePullSecrets with fallback to app secrets - Merge standalone compose.example.pipelock.yml into compose.example.ai.yml - Add pipelock.example.yaml for Docker Compose users - Add exclude-paths to CI workflow for locale file false positives * Add external assistant support (OpenAI-compatible SSE proxy) Allow self-hosted instances to delegate chat to an external AI agent via an OpenAI-compatible streaming endpoint. Configurable per-family through Settings UI or ASSISTANT_TYPE env override. - Assistant::External::Client: SSE streaming HTTP client (no new gems) - Settings UI with type selector, env lock indicator, config status - Helm chart and Docker Compose env var support - 45 tests covering client, config, routing, controller, integration * Add session key routing, email allowlist, and config plumbing Route to the actual OpenClaw session via x-openclaw-session-key header instead of creating isolated sessions. Gate external assistant access behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var). Plumb session_key and allowedEmails through Helm chart, compose, and env template. * Add HTTPS_PROXY support to External::Client for Pipelock integration Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike Faraday). Explicitly resolve proxy from environment in build_http so outbound traffic to the external assistant routes through Pipelock's forward proxy when enabled. Respects NO_PROXY for internal hosts. * Add UI fields for external assistant config (Setting-backed with env fallback) Follow the same pattern as OpenAI settings: database-backed Setting fields with env var defaults. Self-hosters can now configure the external assistant URL, token, and agent ID from the browser (Settings > Self-Hosting > AI Assistant) instead of requiring env vars. Fields disable when the corresponding env var is set. * Improve external assistant UI labels and add help text Change placeholder to generic OpenAI-compatible URL pattern. Add help text under each field explaining where the values come from: URL from agent provider, token for authentication, agent ID for multi-agent routing. * Add external assistant docs and fix URL help text Add External AI Assistant section to docs/hosting/ai.md covering setup (UI and env vars), how it works, Pipelock security scanning, access control, and Docker Compose example. Drop "chat completions" jargon from URL help text. * Harden external assistant: retry logic, disconnect UI, error handling, and test coverage - Add retry with backoff for transient network errors (no retry after streaming starts) - Add disconnect button with confirmation modal in self-hosting settings - Narrow rescue scope with fallback logging for unexpected errors - Safe cleanup of partial responses on stream interruption - Gate ai_available? on family assistant_type instead of OR-ing all providers - Truncate conversation history to last 20 messages - Proxy-aware HTTP client with NO_PROXY support - Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key) - Full test coverage for streaming, retries, proxy routing, config, and disconnect * Exclude external assistant client from Pipelock scan-diff False positive: `@token` instance variable flagged as "Credential in URL". Temporary workaround until Pipelock supports inline suppression. * Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens - Fix NO_PROXY matching to require domain boundary (exact match or .suffix), case-insensitive. Prevents badexample.com matching example.com. - Add done flag to SSE streaming so read_body stops after [DONE] - Move MAX_CONVERSATION_MESSAGES to class level - Use bg-success/bg-destructive design tokens for status indicators - Add rationale comment for pipelock scan exclusion - Update docs last-updated date * Address second round of review feedback - Allowlist email comparison is now case-insensitive and nil-safe - Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams - Don't expose upstream HTTP response body in user-facing errors (log it instead) - Fix frozen string warning on buffer initialization - Fix "builtin" typo in docs (should be "built-in") * Protect completed responses from cleanup, sanitize error messages - Don't destroy a fully streamed assistant message if post-stream metadata update fails (only cleanup partial responses) - Log raw connection/HTTP errors internally, show generic messages to users to avoid leaking network/proxy details - Update test assertions for new error message wording * Fix SSE content guard and NO_PROXY test correctness Use nil check instead of present? for SSE delta content to preserve whitespace-only chunks (newlines, spaces) that can occur in code output. Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so the proxy resolution and NO_PROXY bypass logic are actually exercised. * Forward proxy credentials to Net::HTTP Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so authenticated proxies (http://user:pass@host:port) work correctly. Without this, credentials parsed from the proxy URL were silently dropped. Nil values are safe as positional args when no creds exist. * Update pipelock integration to v0.3.1 with full scanning config Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security sections to both the Helm ConfigMap and compose example config: mcp_tool_policy, mcp_session_binding, and tool_chain_detection. These protect the /mcp endpoint against tool injection, session hijacking, and multi-step exfiltration chains. Add version and mode fields to config files. Enable include_defaults for DLP and response scanning to merge user patterns with the 35 built-in patterns. Remove redundant --mode CLI flag from the Helm deployment template since mode is now in the config file. |
||
|
MkDev11
|
0afdb1d0fd |
Feature/pdf import transaction rows (#846)
* Add import row generation from PDF extracted data - Add generate_rows_from_extracted_data method to PdfImport - Add import! method to create transactions from PDF rows - Update ProcessPdfJob to generate rows after extraction - Update configured?, cleaned?, publishable? for PDF workflow - Add column_keys, required_column_keys, mapping_steps - Set bank statements to pending status for user review - Add tests for new functionality Closes #844 * Add tests for BankStatementExtractor - Test transaction extraction from PDF content - Test deduplication across chunk boundaries - Test amount normalization for various formats - Test graceful handling of malformed JSON responses - Test error handling for empty/nil PDF content * Fix supports_pdf_processing? to validate effective model The validation was always checking @default_model, but process_pdf allows overriding the model via parameter. This could cause a vision-capable override model to be rejected, or a non-vision-capable override to pass validation only to fail during processing. Changes: - supports_pdf_processing? now accepts optional model parameter - process_pdf passes effective model to validation - Raise Provider::Openai::Error inside with_provider_response for consistent error handling Addresses review feedback from PR#808 * Fix insert_all! bug: explicitly set import_id Rails insert_all! on associations does NOT auto-set the foreign key. Added import_id explicitly and use Import::Row.insert_all! directly. Also reload rows before counting to ensure accurate count. * Fix pending status showing as processing for bank statements with rows When bank statement PDF imports have extracted rows, show a 'Ready for Review' screen with a link to the confirm path instead of the 'Processing' spinner. This addresses the PR feedback that users couldn't reach the review flow even though rows were created. * Gate publishable? on account.present? to prevent import failure PDF imports are created without an account, and import! raises if account is missing. This prevents users from hitting publish and having the job fail. * Wrap generate_rows_from_extracted_data in transaction for atomicity - Clear rows and reset count even when no transactions extracted - Use transaction block to prevent partial updates on failure - Use mapped_rows.size instead of reload for count * Localize transactions count string with i18n helper * Add AccountMapping step for PDF imports when account is nil PDF imports need account selection before publishing. This adds Import::AccountMapping to mapping_steps when account is nil, matching the behavior of TransactionImport and TradeImport. Addresses PR#846 feedback about account selection for PDF imports. * Only include CategoryMapping when rows have non-empty categories PDF extraction doesn't extract categories from bank statements, so the CategoryMapping step would show empty. Now we only include CategoryMapping if rows actually have non-empty category values. This prevents showing an empty mapping step for PDF imports. * Fix PDF import UI flow and account selection - Add direct account selection in PDF import UI instead of AccountMapping - AccountMapping designed for CSV imports with multiple account values - PDF imports need single account for all transactions - Add update action and route for imports controller - Fix controller to handle pdf_import param format from form_with - Show Publish button when import is publishable (account set) - Fix stepper nav: Upload/Configure/Clean non-clickable for PDF imports - Redirect PDF imports from configuration step (auto-configured) - Improve AI prompt to recognize M-PESA/mobile money as bank statements - Fix migration ordering for import_rows table columns * Add guard for invalid account_id in imports#update Prevents silently clearing account when invalid ID is passed. Returns error message instead of confusing 'Account saved' notice. * Localize step names in import nav and add account guard - Use t() helper for all step names (Upload, Configure, Clean, Map, Confirm) - Add guard for invalid account_id in imports#update - Prevents silently clearing account when invalid ID is passed * Make category column migrations idempotent Check if columns exist before adding to prevent duplicate column errors when migrations are re-run with new timestamps. * Add match_path for PDF import step highlighting Fixes step detection when path is nil by using separate match_path for current step highlighting while keeping links disabled. * Rename category migrations and update to Rails 7.2 - Rename class to EnsureCategoryFieldsOnImportRows to avoid conflicts - Rename class to EnsureCategoryIconOnImportRows - Update migration version from 7.1 to 7.2 per guidelines - Rename files to match class names - Add match_path for PDF import step highlighting * Use primary (black) style for Create Account and Save buttons * Remove match_path from auto-completed PDF steps Only step 4 (Confirm) needs match_path for active-step detection. Steps 1-3 are purely informational and always complete. * Add fallback for document type translation Handles nil or unexpected document_type values gracefully. Also removes match_path from auto-completed PDF steps. * Use index-based step number for mobile indicator Fixes 'Step 5 of 4' issue when Map step is dynamically removed. * Fix hostings_controller_test: use blank? instead of nil Setting returns empty string not nil for unset values. * Localize step progress label and use design token * Fix button styling: use design system Tailwind classes btn--primary and btn--secondary CSS classes don't exist. Use actual design system classes from DS::Buttonish. * Fix CRLF line endings in tags_controller_test.rb --------- Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com> |
||
|
LPW
|
61eb611529 |
Simplefin enhancements v2 (#267)
* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint # Conflicts: # db/schema.rb # Conflicts: # app/controllers/simplefin_items_controller.rb * fix testing * fix linting * xfix linting x2 * Review PR #267 on we-promise/sure (SimpleFin enhancements v2). Address all 15 actionable CodeRabbit comments: Add UUID validations in rakes (e.g., simplefin_unlink), swap Ruby pattern matching/loops for efficient DB queries (e.g., where LOWER(name) LIKE ?), generate docstrings for low-coverage areas (31%), consolidate routes for simplefin_items, move view logic to helpers (e.g., format_transaction_extra), strengthen tests with exact assertions/fixtures for dedup/relink failures. Also, check for overlaps with merged #262 (merchants fix): Ensure merchant creation in simplefin_entry/processor.rb aligns with new payee-based flow and MD5 IDs; add tests for edge cases like empty payees or over-merging pendings. Prioritize security (PII redaction in logs, no hardcoded secrets). * SimpleFin: address CodeRabbit comments (batch 1) - Consolidate simplefin_items routes under a single resources block; keep URLs stable - Replace inline JS with Stimulus auto-relink controller; auto-load relink modal via global modal frame - Improve a11y in relink modal by wrapping rows in labels - Harden unlink rake: default dry_run=true, UUID validation, redact PII in outputs, clearer errors - Backfill rake: default dry_run=true, UUID validation; groundwork for per-SFA counters - Fix-was-merged rake: default dry_run=true, UUID validation; clearer outputs - Idempotent transfer auto-match (find_or_create_by! + RecordNotUnique rescue) - Extract SimpleFin error tooltip assembly into helper and use it in view RuboCop: maintain 2-space indentation, spaces inside array brackets, spaces after commas, and no redundant returns * Linter noise * removed filed commited by mistake. * manual relink flow and tighten composite matching * enforce manual relink UI; fix adapter keywords; guarantee extra.simplefin hash * refactor(simplefin): extract relink service; enforce manual relink UI; tighten composite match; migration 7.2 * add provider date parser; refactor rake; move view queries; partial resilience * run balances-only import in background job. make update flow enqueue balances-only job * persists across all update redirects and initialize used_manual_ids to prevent NameError in relink candidate computation. * SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint * Fixed failed test after rebase. * scan_ruby fix * Calming the rabbit: Fix AccountProvider linking when accounts change Drop the legacy unique index instead of duplicating it Fix dynamic constant assignment Use fixtures consistently; avoid rescue for control flow. Replace bare rescue with explicit exception class. Move business logic out of the view. Critical: Transaction boundary excludes recompute phase, risking data loss. Inconsistency between documentation and implementation for zero-error case. Refactor to use the compute_unlinked_count helper for consistency. Fix cleanup task default: it deletes by default. Move sync stats computation to controller to avoid N+1 queries. Consolidate duplicate sync query. Clarify the intent of setting flash notice on the error path. Fix Date/Time comparison in should_be_inactive?. Move stats retrieval logic to controller. Remove duplicate Sync summary section. Remove the unnecessary sleep statement; use Capybara's built-in waiting. Add label wrappers for accessibility and consistency. * FIX SimpleFIN new account modal Now new account properly loads as a Modal, instead of new page. Fixes also form showing dashboard instead of settings page. * Remove SimpleFin legacy UI components, migrate schema, and refine linking behavior. # Conflicts: # app/helpers/settings_helper.rb * Extract SimpleFin-related logic to `prepare_show_context` helper and refactor for consistency. Adjust conditional checks and ensure controller variables are properly initialized. * Remove unused SimpleFin maps from prepare_show_context; select IDs to avoid N+1 Replace Tailwind bg-green-500 with semantic bg-success in _simplefin_panel/_provider_form Add f.label :setup_token in simplefin_items/new for a11y Remove duplicate require in AccountsControllerSimplefinCtaTest * Remove unnecessary blank lines * Reduce unnecessary changes This reduces the diff against main * Simplefin Account Setup: Display in modal This fixes an issue with the `X` dismiss button in the top right corner * Removed unnecessary comment. * removed unnecessary function. * fixed broken links * Removed unnecessary file * changed to database query * set to use UTC and gaurd against null * set dry_run=true * Fixed comment * Changed to use a database-level query * matched test name to test behavior. * Eliminate code duplication and Time.zone dependency * make final summary surface failures * lint fix * Revised timezone comment. better handle missing selectors. * sanitized LIKE wildcards * Fixed SimpleFin import to avoid “Currency can’t be blank” validation failures when providers return an empty currency string. * Added helper methods for admin and self-hosted checks * Specify exception types in rescue clauses. * Refined logic to determine transaction dates for credit accounts. * Refined stats calculation for `total_accounts` to track the maximum unique accounts per run instead of accumulating totals. * Moved `unlink_all!` logic to `SimplefinItem::Unlinking` concern and deprecated `SimplefinItem::Unlinker`. Updated related references. * Refined legacy unlinking logic, improved `current_holdings` formatting, and added ENV-based overrides for self-hosted checks. * Enhanced `unlink_all!` with explicit error handling, improved transaction safety, and refined ENV-based self-hosted checks. Adjusted exception types and cleaned up private method handling. * Improved currency assignment logic by adding fallback to `current_account` and `family` currencies. * Enhanced error tracking during SimpleFin account imports by adding categorized error buckets, limiting stored errors to the last 5, and improving `stats` calculations. * typo fix * Didn't realize rabbit was still mad... Refactored SimpleFin error handling and CTA logic: centralized duplicate detection and relink visibility into controller, improved task counters, adjusted redirect notices, and fixed form indexing. * Dang rabbit never stops... Centralized SimpleFin maps logic into `MapsHelper` concern and integrated it into relevant controllers and rake tasks. Optimized queries, reduced redundancy, and improved unlinked counts and manual account checks with batch processing. Adjusted task arguments for clarity. * Persistent rabbit. Optimized SimpleFin maps logic by implementing batch queries for manual account and unlinked count checks, reducing N+1 issues. Improved clarity of rake task argument descriptions and error messages for better usability. * Lost a commit somehow, resolved here. Refactored transaction extra details logic by introducing `build_transaction_extra_details` helper to improve clarity, reusability, and reduce view complexity. Enhanced rake tasks with strict dry-run validation and better error handling. Updated schema to allow nullable `merchant_id` and added conditional unique indexes for recurring transactions. * Refactored sensitive data redaction in `simplefin_unlink` task for recursive handling, optimized SQL sanitization in `simplefin_holdings_backfill`, improved error handling in `transactions_helper`, and streamlined day change calculation logic in `Holding` model. * Lint fix * Removed per PR comments. * Also removing per PR comment. * git commit -m "SimpleFIN polish: preserve #manual-accounts wrapper, unify \"manual\" scope, and correct unlinked counts - Preserve #manual-accounts wrapper: switch non-empty updates to turbo_stream.update and background broadcast_update_to; keep empty-path replace to render <div id=\"manual-accounts\"></div> - Unify definition of manual accounts via Account.visible_manual (visible + legacy-nil + no AccountProvider); reuse in controllers, jobs, and helper - Correct setup/unlinked counts: SimplefinItem::Syncer#finalize_setup_counts and maps now consider AccountProvider links (legacy account AND provider must be absent) Deleted: - app/models/simplefin_item/relink_service.rb - app/controllers/concerns/simplefin_items/relink_helpers.rb - app/javascript/controllers/auto_relink_controller.js - app/views/simplefin_items/_relink_modal.html.erb - app/views/simplefin_items/manual_relink.html.erb - app/views/simplefin_items/relink.html.erb - test/services/simplefin_item/relink_service_test.rb Refs: PR #318 unified link/unlink; PR #267 SimpleFIN; follow-up to fix wrapper ID loss and counting drift." * Extend unlinked account check to include "Investment" type * set SimpleFIN item for `balances`, remove redundant unpacking, and improve holdings task error * SimpleFIN: add `errors` action + modal; do not reintroduce legacy relink actions; removed dead helper * FIX simpleFIN linking * Add delay back, tests benefit from it * Put cache back in * Remove empty `rake` task * Small spelling fixes. --------- Signed-off-by: soky srm <sokysrm@gmail.com> Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: sokie <sokysrm@gmail.com> Co-authored-by: Dylan Corrales <deathcamel58@gmail.com> |
||
|
João Felipe
|
9fefe57de5 |
Feature/yahoo finance (#123)
* Implement Yahoo Finance * Added tests * Updated hosting controller to check for managed app_mode instead of env_override * Suggestions from CodeRabbit and Fixes on tests * Remove Css changes * Fix yahoo finance impl and i18n * Updated view to use healthy method * remove usage * Updated env example * keep usage on class just to keep same format * Ci test * Remove some useless validations * Remove logs * Linter fixes * Broke this in my conflict merge * Wrong indentation level --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Juan José Mata
|
72e7d7736b |
Add onboarding state selector for self-hosted signup (#251)
* Add onboarding modes to self-hosted signup * Style form consistently * Configure ONBOARDING_STATE via ENV |
||
|
soky srm
|
8cd109a5b2 |
Implement support for generic OpenAI api (#213)
* Implement support for generic OpenAI api - Implements support to route requests to any openAI capable provider ( Deepsek, Qwen, VLLM, LM Studio, Ollama ). - Keeps support for pure OpenAI and uses the new better responses api - Uses the /chat/completions api for the generic providers - If uri_base is not set, uses default implementation. * Fix json handling and indentation * Fix linter error indent * Fix tests to set env vars * Fix updating settings * Change to prefix checking for OAI models * FIX check model if custom uri is set * Change chat to sync calls Some local models don't support streaming. Revert to sync calls for generic OAI api * Fix tests * Fix tests * Fix for gpt5 message extraction - Finds the message output by filtering for "type" == "message" instead of assuming it's at index 0 - Safely extracts the text using safe navigation operators (&.) - Raises a clear error if no message content is found - Parses the JSON as before * Add more langfuse logging - Add Langfuse to auto categorizer and merchant detector - Fix monitoring on streaming chat responses - Add Langfuse traces also for model errors now * Update app/models/provider/openai.rb Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: soky srm <sokysrm@gmail.com> * handle nil function results explicitly * Exposing some config vars. * Linter and nitpick comments * Drop back to `gpt-4.1` as default for now * Linter * Fix for strict tool schema in Gemini - This fixes tool calling in Gemini OpenAI api - Fix for getTransactions function, page size is not used. --------- Signed-off-by: soky srm <sokysrm@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Juan José Mata
|
5d6915a994 |
Add OpenAI token configuration to self-hosting settings (#122)
* feat: add OpenAI settings partial |
||
|
Juan José Mata
|
54bc37a651 |
Remove Synth Finance references (#47)
* Remove Synth Finance integration * Linter noise * Fix failing (old) test, use it for Twelve Data --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Vincent Teo
|
5bdefe6e63 |
feat: Add Twelve Data provider for exchange rates and securities (#2)
* feat: Add Twelve Data provider for exchange rates and securities * test: fix hosting controller test, linting * fix: add countries gem to handle country codes in Twelve Data provider * fix: allow security search combobox to have no logo * refactor: update Twelve Data provider use time series endpoint * fix: set twelve data as default provider |
||
|
Zach Gollwitzer
|
e657c40d19 |
Account:: namespace simplifications and cleanup (#2110)
* Flatten Holding model * Flatten balance model * Entries domain renames * Fix valuations reference * Fix trades stream * Fix brakeman warnings * Fix tests * Replace existing entryable type references in DB |
||
|
Zach Gollwitzer
|
2f6b11c18f |
Personal finance AI (v1) (#2022)
* AI sidebar * Add chat and message models with associations * Implement AI chat functionality with sidebar and messaging system - Add chat and messages controllers - Create chat and message views - Implement chat-related routes - Add message broadcasting and user interactions - Update application layout to support chat sidebar - Enhance user model with initials method * Refactor AI sidebar with enhanced chat menu and interactions - Update sidebar layout with dynamic width and improved responsiveness - Add new chat menu Stimulus controller for toggling between chat and chat list views - Improve chat list display with recent chats and empty state - Extract AI avatar to a partial for reusability - Enhance message display and interaction styling - Add more contextual buttons and interaction hints * Improve chat scroll behavior and message styling - Refactor chat scroll functionality with Stimulus controller - Optimize message scrolling in chat views - Update message styling for better visual hierarchy - Enhance chat container layout with flex and auto-scroll - Simplify message rendering across different chat views * Extract AI avatar to a shared partial for consistent styling - Refactor AI avatar rendering across chat views - Replace hardcoded avatar markup with a reusable partial - Simplify avatar display in chats and messages views * Update sidebar controller to handle right panel width dynamically - Add conditional width class for right sidebar panel - Ensure consistent sidebar toggle behavior for both left and right panels - Use specific width class for right panel (w-[375px]) * Refactor chat form and AI greeting with flexible partials - Extract message form to a reusable partial with dynamic context support - Create flexible AI greeting partial for consistent welcome messages - Simplify chat and sidebar views by leveraging new partials - Add support for different form scenarios (chat, new chat, sidebar) - Improve code modularity and reduce duplication * Add chat clearing functionality with dynamic menu options - Implement clear chat action in ChatsController - Add clear chat route to support clearing messages - Update AI sidebar with dropdown menu for chat actions - Preserve system message when clearing chat - Enhance chat interaction with new menu options * Add frontmatter to project structure documentation - Create initial frontmatter for structure.mdc file - Include description and configuration options - Prepare for potential dynamic documentation rendering * Update general project rules with additional guidelines - Add rule for using `Current.family` instead of `current_family` - Include new guidelines for testing, API routes, and solution approach - Expand project-specific rules for more consistent development practices * Add OpenAI gem and AI-friendly data representations - Add `ruby-openai` gem for AI integration - Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement - Include Promptable module in both models - Add savings rate calculation method in IncomeStatement - Prepare financial models for AI-powered insights and interactions * Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities - Implement comprehensive AI financial query system with function-based interactions - Add detailed debug logging for AI responses and function calls - Extend BalanceSheet and IncomeStatement models with AI-friendly methods - Create robust error handling and fallback mechanisms for AI queries - Update chat and message views to support debug mode and enhanced rendering - Add AI query routes and initial test coverage for financial assistant * Refactor AI sidebar and chat layout with improved structure and comments - Remove inline AI chat from application layout - Enhance AI sidebar with more semantic HTML structure - Add descriptive comments to clarify different sections of chat view - Improve flex layout and scrolling behavior in chat messages container - Optimize message rendering with more explicit class names and structure * Add Markdown rendering support for AI chat messages - Implement `markdown` helper method in ApplicationHelper using Redcarpet - Update message view to render AI messages with Markdown formatting - Add comprehensive Markdown rendering options (tables, code blocks, links) - Enhance AI Financial Assistant prompt to encourage Markdown usage - Remove commented Markdown CSS in Tailwind application stylesheet * Missing comma * Enhance AI response processing with chat history context * Improve AI debug logging with payload size limits and internal message flag * Enhance AI chat interaction with improved thinking indicator and scrolling behavior * Add AI consent and enable/disable functionality for AI chat * Upgrade Biome and refactor JavaScript template literals - Update @biomejs/biome to latest version with caret (^) notation - Refactor AI query and chat controllers to use template literals - Standardize npm scripts formatting in package.json * Add beta testing usage note to AI consent modal * Update test fixtures and configurations for AI chat functionality - Add family association to chat fixtures and tests - Set consistent password digest for test users - Enable AI for test users - Add OpenAI access token for test environment - Update chat and user model tests to include family context * Simplify data model and get tests passing * Remove structure.mdc from version control * Integrate AI chat styles into existing prose pattern * Match Figma design spec, implement Turbo frames and actions for chats controller * AI rules refresh * Consolidate Stimulus controllers, thinking state, controllers, and views * Naming, domain alignment * Reset migrations * Improve data model to support tool calls and message types * Tool calling tests and fixtures * Tool call implementation and test * Get assistant test working again * Test updates * Process tool calls within provider * Chat UI back to working state again * Remove stale code * Tests passing * Update openai class naming to avoid conflicts * Reconfigure test env * Rebuild gemfile * Fix naming conflicts for ChatResponse * Message styles * Use OpenAI conversation state management * Assistant function base implementation * Add back thinking messages, clean up error handling for chat * Fix sync error when security price has bad data from provider * Add balance sheet function to assistant * Add better function calling error visibility * Add income statement function * Simplify and clean up "thinking" interactions with Turbo frames * Remove stale data definitions from functions * Ensure VCR fixtures working with latest code * basic stream implementation * Get streaming working * Make AI sidebar wider when left sidebar is collapsed * Get tests working with streaming responses * Centralize provider error handling * Provider data boundaries --------- Co-authored-by: Josh Pigford <josh@joshpigford.com> |
||
|
Zach Gollwitzer
|
19cc63c8f4 |
Use Redis for ActiveJob and ActionCable (#2004)
* Use Redis for ActiveJob and ActionCable * Fix alwaysApply setting * Update queue names and weights * Tweak weights * Update job queues * Update docker setup guide * Remove deprecated upgrade columns from users table * Refactor Redis configuration for Sidekiq and caching in production environment * Add Sidekiq Sentry monitoring * queue naming fix * Clean up schema |
||
|
Zach Gollwitzer
|
f65b93a352 |
Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control * Simplification of providers interface * Synth tests * Update money to use new find rates method * Remove unused issues code * Additional issue feature removals * Update price data fetching and tests * Update documentation for providers * Security test fixes * Fix self host test * Update synth usage data access * Remove AI pr schema changes |
||
|
Tony Vincent
|
8208722247 |
Feat: Data "reset" button (#1913)
* feat: Allow admins to delete family data * feat: Allow self-hosting users to delete cached data * Remove system tests |
||
|
Zach Gollwitzer
|
7fabca4679 | Simplify self host settings controller (#1230) | ||
|
Zach Gollwitzer
|
fa08f027c7 |
Sync notifications and troubleshooting guides (#998)
* Add help articles * Broadcast sync messages as notifications * Lint fixes * more lint fixes * Remove redundant code |
||
|
Zach Gollwitzer
|
5dfbba403a |
Test environment stability improvements (#703)
* Add climate_control gem and test helper * Replace ENV mods in upgrades test * Replace ENV mods in registrations test * Remove ENV references in hostings controller * Update ENV refs in mailer test * ActiveStorage cleanup * Consolidate queue config so appropriate adapter runs in test environment * Make test environment more explicit * Centralize self hosting config * Remove flaky system test |
||
|
Thibaut Gorioux
|
6fdb8e8d69 |
Allow a self-hosted user to configure their SMTP settings directly from within the UI (#682)
* Add setting fields to model * Allow to configure SMTP settings * Normalize locales * Cleanup locales * Remove 'coming soon' * fix test * Reset credentials * Reset development config * Check smtp spelling * Use post instead of get method * TLS ENV variable is more descriptive * Rework application mailer * Follow rails convention for mailer action params * Reset schema.rb to main * Test WIP * Add test for controller and mailer * Move tests from controller to model * Custom error message if settings are not all present * Comment smtp config in development env * Add default tls enabled value * Rubocop * Fix controller test * Reset credentials * Normalize locales * Test * fix test * Fix application mailer test that fails randomly * Error flash message instead of notice * Rework application mailer tests |
||
|
Zach Gollwitzer
|
8ea7b54fe8 |
Update self host settings page styles (#677)
* Update page styles * Add new styles to self host settings page * Update self hosting page title |
||
|
Zach Gollwitzer
|
9bda7efc3f |
New Settings Menu, Routes and Controllers Organization (#641)
* Add new settings routes and controllers * Add new settings view, restructure controllers and routes * Fix lint errors |