7 Commits

Include renames

Author	SHA1	Message	Date
Brandon	25a9011f14	feat(ai): analytical tool-set upgrade for the builtin assistant (#3064) ... * fix(assistant): survive tool failures with error and hint results instead of aborting the turn A tool exception used to raise FunctionExecutionError out of the responder loop, turning the whole turn into a generic chat error banner. An unknown tool name was worse: the rescue block itself crashed (fn.name on nil). Tool failures now come back to the model as data ({error, hint}) so the conversation survives and the model can retry once with corrected arguments. The catch-all branch logs and tells the model not to retry. FunctionExecutionError remains defined for API compatibility. * fix(assistant): strict schemas declare every property as required get_categories and get_tags declared an optional page property while inheriting strict mode, which is invalid under strict function calling (every property must be listed in required). Both now opt out of strict mode like every other paginated tool, and gain a page_size param (1..100) while being touched. A registry-walking test asserts the invariant for every current and future tool, preview tools included. * fix(assistant): HistoryTrimmer always keeps the newest turn Trimming iterates newest-first and stopped at the first group over budget. When the newest group alone exceeded the budget, everything was dropped, including the user message the model was being asked to answer, and the provider received only the system prompt. The newest group now always survives. * perf(assistant): compact AI time series and make account history opt-in get_accounts shipped a 5-year monthly series for every account on every call, roughly 60 formatted money strings per account, which dominates the tool payload for multi-account families and swamps small self-hosted context windows. The series is now opt-in (include_balance_series) and bounded by a named period (series_period, default last_365_days). to_ai_time_series states the currency once and emits numeric values instead of formatting every point; the system prompt already tells the model how to render currency. get_accounts also now returns account ids (they are what other tools accept as account_ids filters) and respects the visible scope, so hidden accounts no longer leak into responses. * refactor(assistant): id and name filters replace user-data enums in get_transactions The schema inlined every account, category, merchant, and tag name as enum values on every request. That grows without bound with family data, defeats provider prompt caching (definitions change whenever a name does), and is the pattern that made empty-enum pruning necessary in the first place. Filters are now plain string arrays documented as exact names from the sibling get_* tools, which Transaction::Search already resolves server-side, plus an account_ids UUID filter. New params: page_size (1..100), sort_by amount, types (income/expense/transfer, the way to exclude transfers), and statuses (pending/confirmed). The three now-unused enum helpers are removed from the base class; family_tag_names stays for update_tag, which still identifies tags by name. * feat(assistant): add get_merchants and get_recurring_transactions Merchants were unreachable: names appeared nowhere and update_transaction's merchant_id had no source of ids, making it unusable. get_merchants lists id, exact name, and source, scoped through available_merchants_for so merchants seen only in accounts hidden from the user never leak. Recurring transactions had a model, an Upcoming view, and no assistant reach. get_recurring_transactions lists detected and manual recurring items (status filter defaulting to active, optional upcoming_within_days window) with per-currency totals of active non-transfer items, answering subscription and upcoming-bill questions directly instead of via transaction paging. * feat(assistant): flexible periods on get_balance_sheet and trends on get_income_statement get_balance_sheet was hard-wired to five years of monthly history with no parameters, although Period supports arbitrary ranges and the chart builder takes any interval. It now accepts a named period or custom dates plus an interval, with a 400-point cap so a day-granularity request over a decade returns an error instead of a giant series. The default call is byte-compatible with the old shape. The balance sheet object is also memoized; it was being constructed four times per call. get_income_statement gains the analysis surface the assistant lacked: group_by month for a monthly income/expenses/net series (capped at 36 buckets), compare_previous_period for an equal-length prior window with absolute and percent deltas, and account_ids to scope totals to specific accounts via IncomeStatement#totals_for. Category breakdowns are family-wide by construction, so the account-filtered view omits them and says why. Unknown or inaccessible account ids come back as a soft failure naming the ids so the model can correct itself. * feat(assistant): preview reads for insights and valuations The Insights feed is generated nightly with pre-computed numbers, and the chat assistant could not read a word of it. get_insights returns the visible feed (type filter, acknowledged toggle, limit) without marking anything read; an assistant read is not the user viewing the feed. It sits in PREVIEW_FUNCTION_CLASSES because the feature itself is preview-gated, which also keeps it off the default /mcp surface. record_valuation was write-only: an agent recording provenance-cited valuations had no way to audit what it wrote or find dates already carrying a value. get_valuations lists valuation entries newest first with kind and the citation notes, scoped to accessible visible accounts. * feat(assistant): cache-stable system prompt with session context The prompt interpolated currency mid-text and the date near the end, so no two requests shared a cacheable prefix, and it told the model nothing about the family: not one account name, not a single category. Models opened most conversations blind, either wandering through tools or answering without data. The prompt is now STATIC_INSTRUCTIONS, a frozen constant that is byte-identical for every request (providers discount an exactly-repeated prefix; tool definitions are also stable now that schemas carry no user data), followed by a trailing Session context block holding everything volatile: date, date format, currency details, an account roster with balances, and category names. The static half gains a request-classification rule (CHAT / LOOKUP / ANALYSIS), a reuse-what-you-have rule with an explicit re-fetch carve-out, specific-tool preference, and the error/hint retry-once rule that pairs with the tool soft-fail contract. Context stays cheap by construction: the roster collapses to per-type counts beyond 25 accounts, categories to a count beyond 60 names, and both collapse whenever the configured context window is under 4096 (the self-hosted default is 2048), via the new Assistant::TokenBudget helper. Intro chats are untouched. * feat(assistant): raise tool-round cap to 8 with a no-tools grace turn; instructions-aware history budget Five rounds was tight for a tool surface that now supports real analysis chains, and hitting the cap raised ToolCallLimitError, which surfaced to the user as a dead chat with an error banner. The default is now eight rounds (env override unchanged), and on the final permitted round the follow-up request offers no tools, so the model must answer in text with whatever it gathered. The limit error remains as a defensive backstop. The generic-path history budget reserved a flat 256 tokens for a system prompt that already estimates well past that; the trimmer now budgets against the actual instructions when available. LLM_MAX_RESPONSE_TOKENS was reserved in budget math but never sent to the provider. It is now sent (max_tokens on chat completions, max_output_tokens on the Responses API) only when explicitly configured via ENV or a stored Setting; stock installs keep today's uncapped behavior. * test(evals): chat golden v2 exercising the real prompt and registry The eval runner scored a fiction: hardcoded instructions and four fake permissive tool schemas, so a prompt or registry regression could sail through green. It now runs STATIC_INSTRUCTIONS plus a fixed synthetic session context and builds definitions from Assistant.function_classes against a reference user (classes whose schema cannot build are skipped with a log line, never faked). chat_golden_v2 adds routing scenarios the upgrade cares about: CHAT classification must use no tools, aggregates route to get_income_statement / get_balance_sheet rather than transaction paging, and the new analytical tools are selected with sensible params. The dataset header documents the harness's single-shot limitation. * docs(ai,mcp): current tool tables, responder loop, prompt structure, timeout math Both docs listed 7 tools against a registry of 19, in three separate drift-prone copies. mcp.md now carries the canonical tables (default + preview); ai.md links to them from the MCP section, keeps one grouped functions list for the architecture chapter, and replaces its stale hardcoded registry snippet with a pointer to assistant.rb. The architecture section gains the contracts contributors need when adding a function: the responder loop (rounds vs calls, cap 8, the no-tools grace turn) and the error/hint soft-failure convention, plus the prompt's static/session-context split and its collapse gates. Timeout guidance is recomputed for the new default cap. * fix(assistant): address automated review findings Codex and CodeRabbit findings on the initial push, all verified before changing anything: - AI time series rounded every value to two decimals, which turns 0.001 BTC into 0.0; values now round to the currency's own precision (BTC 8, CLF 4, OMR 3). - get_income_statement validated account_ids against all visible accounts, but totals_for excludes hidden, excluded-from-reports and tax-advantaged accounts, so those ids produced silent zeros. Ids now validate against income_statement.eligible_accounts and the soft failure explains eligibility. - get_recurring_transactions computed totals from the displayed rows, so past the 200-row cap the value labeled a total was partial. Totals now aggregate over the full filtered scope in SQL, and the response carries total_results and a truncated flag. The upcoming_within_days window also starts at today, matching its documentation; overdue items appear in unwindowed calls. - get_valuations silently dropped a malformed date filter and presented unfiltered data as filtered; malformed dates now return invalid_date. - get_balance_sheet returned a generic failure for a reversed custom range because Period's own validation raises past the Date::Error rescue; it now returns the structured invalid_date error. - get_insights documents that its family-wide scope matches the web feed exactly (InsightsController serves Current.family.insights to every member), so the tool exposes nothing the /insights page does not already show the same user. - Tests: limit clamp proven against more insights than the cap, Setting fallbacks stubbed in the provider budget tests, currency precision and reversed-range regression tests added. * refactor(assistant): apply reviewer nitpicks - order declares type alongside its enum, matching sort_by - page-size clamp deduplicated into the base class (MAX_PAGE_SIZE + shared resolved_page_size); dead per-tool copies removed - get_accounts preloads balance rows only when the series is requested - get_income_statement validates the bucket count before running any aggregation work Deliberately unchanged: the balance sheet's monthly_history key. The default response shape stays byte-compatible for existing MCP consumers, and the nested series already states its interval. * fix(assistant): second-round review findings on get_valuations - A reversed date range (start after end) now returns the structured invalid_date error instead of presenting an empty result as filtered data, matching get_balance_sheet's handling. - Page numbers are normalized before pagination: Pagy raises on zero, negative or non-numeric pages. The fix lands as a shared resolved_page helper on the base class and applies to every paginated tool (categories, tags, merchants, transactions, holdings, valuations), since all shared the same page-or-1 pattern; schemas declare minimum: 1. * fix(assistant): round series amounts as BigDecimal before Float conversion Converting to Float first can perturb the value at the requested precision; round the exact decimal, then convert for JSON. * fix(assistant): address maintainer review findings - get_accounts no longer fails the whole listing when one account's start date lies beyond the requested period (start_date derives from the first entry, which can be future-dated); that account simply has no series. The unrescued Period.custom was reachable exactly there. - The balances preload is gone: the series goes through Balance::ChartSeriesBuilder, which runs its own query keyed by account ids, so the eager-loaded rows were loaded and discarded. - Provider::Openai#context_window now delegates to Assistant::TokenBudget, removing the duplicated ENV > Setting > default precedence so prompt assembly and the provider can never disagree about the window. * fix(ai): final no-tools round uses tool_choice none instead of dropping tools Anthropic rejects requests whose messages contain tool_use blocks when no tools are defined, so re-requesting with an empty tool list made the final-round grace die in a provider 400 on Anthropic models. The final round now sends the real tool definitions with tool_choice none, which both providers accept, and the model answers in prose as intended. * fix(assistant): scope every income statement read to the requesting user get_income_statement validated account_ids against the user-scoped statement but computed every total from an unscoped one. IncomeStatement falls back to Current.user, which is nil in the assistant job and the MCP endpoint, so the unscoped reads dropped the included_in_finances_for filter and reported family-wide totals next to ids that had been checked against a narrower set. Route all reads through one memoized user-scoped statement, the idiom get_balance_sheet already uses. Also lets the per-instance memoization in IncomeStatement apply across the eligibility check and the totals. Adds a regression test that fails without the change, plus a companion test asserting eligibility and totals agree on scope. Guard the strictness walk against an empty registry so it cannot silently assert nothing.	2026-08-21 21:58:47 +02:00
Brandon	1973c557e5	fix(ai): drop empty data-driven enums from assistant function schemas (#3016) ... * fix(ai): drop empty data-driven enums from assistant function schemas Enum values in tool schemas are built from family data (account names, categories, merchants, tags, tickers). A family with none of these gets enum: [], which is invalid JSON Schema. OpenAI tolerates it, but strict OpenAI-compatible providers reject the entire request, breaking chat for fresh families until they create a tag or merchant. Prune empty enums in build_schema, falling back to a plain string. One choke point covers every function and both consumers: chat tool definitions for all providers, and the /mcp endpoint's tools/list. * fix(ai): address review feedback on enum pruning Stop recursion at populated enum values: enum members are literal values, not subschemas, so a literal like enum: [{ enum: [] }] must be preserved verbatim rather than rewritten. Also cover PREVIEW_FUNCTION_CLASSES in the registry regression test by enabling the preview preference on the test user, with a guard assertion so the test fails if preview functions ever silently drop out.	2026-08-14 02:59:59 +02:00
Blaž Dular	974f1d8e44	feat(mcp): add get/create/update tools for tags and categories (#2374) ... * feat(mcp): add get/create/update tools for tags and categories * test(mcp): add tests for tag and category assistant functions * fix(mcp): harden category/tag assistant functions against bad tool input * refactor(mcp): use UuidFormat.valid? instead of local UUID_PATTERN * chore(mcp): remove account_filter component files that don't belong in this branch	2026-06-21 12:01:39 +02:00
soky srm	9410e5b38d	Providers sharing (#1273) ... * third party provider scoping * Simplify logic and allow only admins to mange providers * Broadcast fixes * FIX tests and build * Fixes * Reviews * Scope merchants * DRY fixes	2026-03-25 17:47:04 +01:00
Zach Gollwitzer	662f2c04ce	Multi-step account forms + clearer balance editing (#2427) ... * Initial multi-step property form * Improve form structure, add optional tooltip help icons to form fields * Add basic inline alert component * Clean up and improve property form lifecycle * Implement Account status concept * Lint fixes * Remove whitespace * Balance editing, scope updates for account * Passing tests * Fix brakeman warning * Remove stale columns * data constraint tweaks * Redundant property	2025-07-03 09:33:07 -04:00
Zach Gollwitzer	5cf758bd03	improvements(ai): Improve AI streaming UI/UX interactions + better separation of AI provider responsibilities (#2039) ... * Start refactor * Interface updates * Rework Assistant, Provider, and tests for better domain boundaries * Consolidate and simplify OpenAI provider and provider concepts * Clean up assistant streaming * Improve assistant message orchestration logic * Clean up "thinking" UI interactions * Remove stale class * Regenerate VCR test responses	2025-04-01 07:21:54 -04:00
Zach GollwitzerandJosh Pigford	2f6b11c18f	Personal finance AI (v1) (#2022) ... * AI sidebar * Add chat and message models with associations * Implement AI chat functionality with sidebar and messaging system - Add chat and messages controllers - Create chat and message views - Implement chat-related routes - Add message broadcasting and user interactions - Update application layout to support chat sidebar - Enhance user model with initials method * Refactor AI sidebar with enhanced chat menu and interactions - Update sidebar layout with dynamic width and improved responsiveness - Add new chat menu Stimulus controller for toggling between chat and chat list views - Improve chat list display with recent chats and empty state - Extract AI avatar to a partial for reusability - Enhance message display and interaction styling - Add more contextual buttons and interaction hints * Improve chat scroll behavior and message styling - Refactor chat scroll functionality with Stimulus controller - Optimize message scrolling in chat views - Update message styling for better visual hierarchy - Enhance chat container layout with flex and auto-scroll - Simplify message rendering across different chat views * Extract AI avatar to a shared partial for consistent styling - Refactor AI avatar rendering across chat views - Replace hardcoded avatar markup with a reusable partial - Simplify avatar display in chats and messages views * Update sidebar controller to handle right panel width dynamically - Add conditional width class for right sidebar panel - Ensure consistent sidebar toggle behavior for both left and right panels - Use specific width class for right panel (w-[375px]) * Refactor chat form and AI greeting with flexible partials - Extract message form to a reusable partial with dynamic context support - Create flexible AI greeting partial for consistent welcome messages - Simplify chat and sidebar views by leveraging new partials - Add support for different form scenarios (chat, new chat, sidebar) - Improve code modularity and reduce duplication * Add chat clearing functionality with dynamic menu options - Implement clear chat action in ChatsController - Add clear chat route to support clearing messages - Update AI sidebar with dropdown menu for chat actions - Preserve system message when clearing chat - Enhance chat interaction with new menu options * Add frontmatter to project structure documentation - Create initial frontmatter for structure.mdc file - Include description and configuration options - Prepare for potential dynamic documentation rendering * Update general project rules with additional guidelines - Add rule for using `Current.family` instead of `current_family` - Include new guidelines for testing, API routes, and solution approach - Expand project-specific rules for more consistent development practices * Add OpenAI gem and AI-friendly data representations - Add `ruby-openai` gem for AI integration - Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement - Include Promptable module in both models - Add savings rate calculation method in IncomeStatement - Prepare financial models for AI-powered insights and interactions * Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities - Implement comprehensive AI financial query system with function-based interactions - Add detailed debug logging for AI responses and function calls - Extend BalanceSheet and IncomeStatement models with AI-friendly methods - Create robust error handling and fallback mechanisms for AI queries - Update chat and message views to support debug mode and enhanced rendering - Add AI query routes and initial test coverage for financial assistant * Refactor AI sidebar and chat layout with improved structure and comments - Remove inline AI chat from application layout - Enhance AI sidebar with more semantic HTML structure - Add descriptive comments to clarify different sections of chat view - Improve flex layout and scrolling behavior in chat messages container - Optimize message rendering with more explicit class names and structure * Add Markdown rendering support for AI chat messages - Implement `markdown` helper method in ApplicationHelper using Redcarpet - Update message view to render AI messages with Markdown formatting - Add comprehensive Markdown rendering options (tables, code blocks, links) - Enhance AI Financial Assistant prompt to encourage Markdown usage - Remove commented Markdown CSS in Tailwind application stylesheet * Missing comma * Enhance AI response processing with chat history context * Improve AI debug logging with payload size limits and internal message flag * Enhance AI chat interaction with improved thinking indicator and scrolling behavior * Add AI consent and enable/disable functionality for AI chat * Upgrade Biome and refactor JavaScript template literals - Update @biomejs/biome to latest version with caret (^) notation - Refactor AI query and chat controllers to use template literals - Standardize npm scripts formatting in package.json * Add beta testing usage note to AI consent modal * Update test fixtures and configurations for AI chat functionality - Add family association to chat fixtures and tests - Set consistent password digest for test users - Enable AI for test users - Add OpenAI access token for test environment - Update chat and user model tests to include family context * Simplify data model and get tests passing * Remove structure.mdc from version control * Integrate AI chat styles into existing prose pattern * Match Figma design spec, implement Turbo frames and actions for chats controller * AI rules refresh * Consolidate Stimulus controllers, thinking state, controllers, and views * Naming, domain alignment * Reset migrations * Improve data model to support tool calls and message types * Tool calling tests and fixtures * Tool call implementation and test * Get assistant test working again * Test updates * Process tool calls within provider * Chat UI back to working state again * Remove stale code * Tests passing * Update openai class naming to avoid conflicts * Reconfigure test env * Rebuild gemfile * Fix naming conflicts for ChatResponse * Message styles * Use OpenAI conversation state management * Assistant function base implementation * Add back thinking messages, clean up error handling for chat * Fix sync error when security price has bad data from provider * Add balance sheet function to assistant * Add better function calling error visibility * Add income statement function * Simplify and clean up "thinking" interactions with Turbo frames * Remove stale data definitions from functions * Ensure VCR fixtures working with latest code * basic stream implementation * Get streaming working * Make AI sidebar wider when left sidebar is collapsed * Get tests working with streaming responses * Centralize provider error handling * Provider data boundaries --------- Co-authored-by: Josh Pigford <josh@joshpigford.com>	2025-03-28 13:08:22 -04:00