Merge pull request #18 from rfc1437/feature/python-worker-queued-runtime

Feature/python worker queued runtime
This commit is contained in:
Georg Bauer
2026-02-23 23:00:15 +01:00
committed by GitHub
26 changed files with 1510 additions and 107 deletions

View File

@@ -10,6 +10,7 @@ When plan and code differ, code is the source of truth.
- [x] Pyodide dependency integrated. - [x] Pyodide dependency integrated.
- [x] Renderer worker runtime exists (`pythonRuntime.worker.ts`) with ready/error/stdout/run protocol. - [x] Renderer worker runtime exists (`pythonRuntime.worker.ts`) with ready/error/stdout/run protocol.
- [x] Runtime timeout watchdog + reset/recovery implemented in `PythonRuntimeManager`. - [x] Runtime timeout watchdog + reset/recovery implemented in `PythonRuntimeManager`.
- [x] Renderer runtime request queueing implemented (concurrent calls are serialized in manager).
- [x] ABI v1 schemas and validation for macro context/result implemented (`abiV1.ts`). - [x] ABI v1 schemas and validation for macro context/result implemented (`abiV1.ts`).
- [x] Benchmark harness implemented (`npm run bench:python-runtime -- <iterations>`). - [x] Benchmark harness implemented (`npm run bench:python-runtime -- <iterations>`).
- [x] Script persistence model implemented (`scripts` DB table + `scripts/*.py` files). - [x] Script persistence model implemented (`scripts` DB table + `scripts/*.py` files).
@@ -18,15 +19,16 @@ When plan and code differ, code is the source of truth.
- [x] Preload + shared API typings for scripts implemented. - [x] Preload + shared API typings for scripts implemented.
- [x] Renderer scripts UX implemented (sidebar list, editor, save, run, delete). - [x] Renderer scripts UX implemented (sidebar list, editor, save, run, delete).
- [x] Script syntax check + entrypoint discovery integrated in editor UX. - [x] Script syntax check + entrypoint discovery integrated in editor UX.
- [x] Blogmark transform pipeline executes Python transform scripts (`kind='transform'`). - [x] Blogmark transform pipeline executes Python transform scripts (`kind='transform'`) via a queued worker runtime by default.
- [x] Project preference `pythonRuntimeMode` added with Settings → Technology section.
## Confirmed Deviations from Original Plan ## Confirmed Deviations from Original Plan
These are current realities and should be treated as authoritative unless we explicitly decide to change them. These are current realities and should be treated as authoritative unless we explicitly decide to change them.
1. **Transform script runtime location differs** 1. **Transform runtime is now configurable (project-level)**
- Original plan: untrusted Python runs in renderer worker only. - Default: `webworker` (worker-thread based Python runtime with queued requests).
- Actual implementation: Blogmark transform scripts run in **main process** Pyodide (`BlogmarkTransformService`). - Optional fallback: `main-thread` legacy execution mode.
2. **Render-time macro migration has not happened yet** 2. **Render-time macro migration has not happened yet**
- Original plan: all render-time macros become Python-backed. - Original plan: all render-time macros become Python-backed.
@@ -36,24 +38,31 @@ These are current realities and should be treated as authoritative unless we exp
- ABI v1 + runtime manager support exist. - ABI v1 + runtime manager support exist.
- Main page generation path still uses existing JS macro rendering. - Main page generation path still uses existing JS macro rendering.
4. **Scripts rebuild/meta-diff sync is still missing** 4. **Scripts rebuild/sync parity is implemented (simple policy)**
- Script CRUD works via app APIs. - `ScriptEngine.rebuildDatabaseFromFiles()` now rebuilds DB metadata from `scripts/*.py`.
- No implemented project-wide “rebuild from files” parity for `scripts/` equivalent to posts/media rebuild flows. - `ScriptEngine.reconcileScriptsFromGitChanges()` now handles added/modified/deleted/renamed script files after git pull.
- Settings → Data now includes **Rebuild Scripts** button (`scripts:rebuildFromFiles`) for manual parity with posts/media rebuild.
## Remaining Work Only ## Remaining Work Only
## 1) Decide and enforce Python runtime boundary (P0) ## 1) Python runtime boundary (P0) — Implemented
- [ ] Decide if `transform` scripts should stay in main process or move to renderer worker. - [x] Worker model introduced for Blogmark transform execution with queued communication.
- [ ] If staying in main process: add explicit timeout/kill/recovery safeguards equivalent to worker watchdog behavior. - [x] Runtime mode made project-configurable via Settings → Technology (`pythonRuntimeMode`).
- [ ] If moving to worker: route transform execution through typed IPC/worker bridge and remove main-process execution path. - [x] Legacy main-thread mode retained as explicit fallback option.
- [ ] Document final security model in this file after decision.
## 2) Add scripts file-system rebuild/sync (P1) ## 2) Add scripts file-system rebuild/sync (P1) — Implemented
- [ ] Implement rebuild/meta-diff style synchronization for `scripts/` so external file edits are detected. - [x] Implement rebuild/meta-diff style synchronization for `scripts/` so external file edits are detected.
- [ ] Define conflict handling policy between DB metadata and script file frontmatter/body. - [x] Define conflict handling policy between DB metadata and script file frontmatter/body.
- [ ] Add tests for create/edit/delete performed outside app while app is closed/open. - [x] Add tests for create/edit/delete performed outside app while app is closed/open.
### Implemented policy (simple)
- Source of truth: script file + docstring frontmatter when present/valid.
- Rebuild path: delete current `scripts` rows for active project and re-import from `scripts/*.py`.
- Reconcile path (git pull): apply file deltas (`added|modified|deleted|renamed`) and upsert/delete rows.
- Conflict behavior: prefer file metadata/body; fall back to safe defaults when values are missing/invalid.
## 3) Wire Python macros into render pipeline (P1) ## 3) Wire Python macros into render pipeline (P1)
@@ -88,7 +97,7 @@ These are current realities and should be treated as authoritative unless we exp
## Acceptance Gate Before Marking Python Scripting “Complete” ## Acceptance Gate Before Marking Python Scripting “Complete”
- [ ] Render-time macros run through Python script path in production generation flow. - [ ] Render-time macros run through Python script path in production generation flow.
- [ ] Scripts directory external changes are synchronized reliably. - [x] Scripts directory external changes are synchronized reliably.
- [ ] Runtime boundary decision implemented and protected by tests. - [x] Runtime boundary decision implemented and protected by tests.
- [ ] Legacy JS macro path removed (or explicitly retained with documented rationale). - [ ] Legacy JS macro path removed (or explicitly retained with documented rationale).
- [ ] `npm test` and `npm run build` pass. - [x] `npm test` and `npm run build` pass.

View File

@@ -0,0 +1,261 @@
import * as path from 'path';
import { Worker } from 'worker_threads';
interface WorkerRunTransformRequest {
type: 'runTransform';
requestId: string;
scriptContent: string;
entrypoint: string;
payloadJson: string;
}
interface WorkerReadyMessage {
type: 'ready';
}
interface WorkerResultMessage {
type: 'transformResult';
requestId: string;
output: unknown;
toasts: string[];
}
interface WorkerErrorMessage {
type: 'transformError';
requestId: string;
error: string;
}
interface WorkerFatalErrorMessage {
type: 'error';
error: string;
}
type WorkerResponseMessage = WorkerReadyMessage | WorkerResultMessage | WorkerErrorMessage | WorkerFatalErrorMessage;
interface QueuedRequest {
request: WorkerRunTransformRequest;
timeoutMs: number;
resolve: (value: { output: unknown; toasts: string[] }) => void;
reject: (error: Error) => void;
}
interface ActiveRequest extends QueuedRequest {
timeoutId: ReturnType<typeof setTimeout> | null;
}
export class BlogmarkPythonWorkerRuntime {
private worker: Worker | null = null;
private workerReady = false;
private workerStartPromise: Promise<void> | null = null;
private workerStartResolve: (() => void) | null = null;
private workerStartReject: ((error: Error) => void) | null = null;
private activeRequest: ActiveRequest | null = null;
private queue: QueuedRequest[] = [];
private requestCounter = 0;
async executeTransform(params: {
scriptContent: string;
entrypoint: string;
payloadJson: string;
timeoutMs?: number;
}): Promise<{ output: unknown; toasts: string[] }> {
const requestId = this.nextRequestId();
const timeoutMs = params.timeoutMs ?? 5000;
return new Promise<{ output: unknown; toasts: string[] }>((resolve, reject) => {
this.queue.push({
request: {
type: 'runTransform',
requestId,
scriptContent: params.scriptContent,
entrypoint: params.entrypoint,
payloadJson: params.payloadJson,
},
timeoutMs,
resolve,
reject,
});
this.dispatchNext().catch((error) => {
reject(error instanceof Error ? error : new Error(String(error)));
});
});
}
dispose(): void {
this.rejectStartPromise(new Error('Python worker runtime disposed'));
this.rejectActiveAndQueue(new Error('Python worker runtime disposed'));
this.resetWorker();
}
private async dispatchNext(): Promise<void> {
if (this.activeRequest || this.queue.length === 0) {
return;
}
await this.ensureWorkerStarted();
const nextRequest = this.queue.shift();
if (!nextRequest) {
return;
}
const timeoutId = setTimeout(() => {
if (!this.activeRequest || this.activeRequest.request.requestId !== nextRequest.request.requestId) {
return;
}
const timeoutError = new Error(`Python transform timed out after ${nextRequest.timeoutMs}ms`);
this.activeRequest.reject(timeoutError);
this.activeRequest = null;
this.resetWorker();
void this.dispatchNext();
}, nextRequest.timeoutMs);
this.activeRequest = {
...nextRequest,
timeoutId,
};
this.worker?.postMessage(nextRequest.request);
}
private async ensureWorkerStarted(): Promise<void> {
if (this.worker && this.workerReady) {
return;
}
if (this.workerStartPromise) {
return this.workerStartPromise;
}
const workerPath = path.join(__dirname, 'blogmarkPython.worker.js');
this.worker = new Worker(workerPath);
this.workerReady = false;
this.worker.on('message', (message: WorkerResponseMessage) => {
this.handleWorkerMessage(message);
});
this.worker.on('error', (error) => {
this.handleWorkerCrash(error instanceof Error ? error : new Error(String(error)));
});
this.worker.on('exit', (code) => {
if (code !== 0) {
this.handleWorkerCrash(new Error(`Python worker exited with code ${code}`));
}
});
this.workerStartPromise = new Promise<void>((resolve, reject) => {
this.workerStartResolve = resolve;
this.workerStartReject = reject;
});
return this.workerStartPromise;
}
private handleWorkerMessage(message: WorkerResponseMessage): void {
if (message.type === 'ready') {
this.workerReady = true;
this.resolveStartPromise();
return;
}
if (message.type === 'error') {
this.handleWorkerCrash(new Error(message.error));
return;
}
const active = this.activeRequest;
if (!active) {
return;
}
if (active.request.requestId !== message.requestId) {
return;
}
if (active.timeoutId) {
clearTimeout(active.timeoutId);
}
this.activeRequest = null;
if (message.type === 'transformResult') {
active.resolve({ output: message.output, toasts: message.toasts });
} else {
active.reject(new Error(message.error));
}
void this.dispatchNext();
}
private handleWorkerCrash(error: Error): void {
this.rejectStartPromise(error);
this.rejectActiveAndQueue(error);
this.resetWorker();
}
private rejectActiveAndQueue(error: Error): void {
if (this.activeRequest) {
if (this.activeRequest.timeoutId) {
clearTimeout(this.activeRequest.timeoutId);
}
this.activeRequest.reject(error);
this.activeRequest = null;
}
while (this.queue.length > 0) {
const queued = this.queue.shift();
queued?.reject(error);
}
}
private resolveStartPromise(): void {
if (this.workerStartResolve) {
this.workerStartResolve();
}
this.workerStartResolve = null;
this.workerStartReject = null;
this.workerStartPromise = null;
}
private rejectStartPromise(error: Error): void {
if (this.workerStartReject) {
this.workerStartReject(error);
}
this.workerStartResolve = null;
this.workerStartReject = null;
this.workerStartPromise = null;
}
private resetWorker(): void {
if (this.worker) {
this.worker.removeAllListeners();
this.worker.terminate();
}
this.worker = null;
this.workerReady = false;
this.workerStartPromise = null;
this.workerStartResolve = null;
this.workerStartReject = null;
}
private nextRequestId(): string {
this.requestCounter += 1;
return `blogmark-py-${this.requestCounter}`;
}
}
let blogmarkPythonWorkerRuntimeInstance: BlogmarkPythonWorkerRuntime | null = null;
export function getBlogmarkPythonWorkerRuntime(): BlogmarkPythonWorkerRuntime {
if (!blogmarkPythonWorkerRuntimeInstance) {
blogmarkPythonWorkerRuntimeInstance = new BlogmarkPythonWorkerRuntime();
}
return blogmarkPythonWorkerRuntimeInstance;
}

View File

@@ -1,5 +1,7 @@
import { z } from 'zod'; import { z } from 'zod';
import { getScriptEngine } from './ScriptEngine'; import { getScriptEngine } from './ScriptEngine';
import { getMetaEngine } from './MetaEngine';
import { getBlogmarkPythonWorkerRuntime } from './BlogmarkPythonWorkerRuntime';
const transformPostSchema = z.object({ const transformPostSchema = z.object({
title: z.string().trim().min(1), title: z.string().trim().min(1),
@@ -55,6 +57,8 @@ export interface BlogmarkTransformResult {
toasts: string[]; toasts: string[];
} }
export type PythonRuntimeMode = 'webworker' | 'main-thread';
const MAX_TOASTS_PER_SCRIPT = 5; const MAX_TOASTS_PER_SCRIPT = 5;
const MAX_TOASTS_TOTAL = 20; const MAX_TOASTS_TOTAL = 20;
const MAX_TOAST_LENGTH = 300; const MAX_TOAST_LENGTH = 300;
@@ -142,6 +146,28 @@ function toErrorMessage(error: unknown): string {
return String(error); return String(error);
} }
function resolveTransformEntrypoint(value: string): string {
const nextEntrypoint = typeof value === 'string' ? value.trim() : '';
if (/^[A-Za-z_][A-Za-z0-9_]*$/.test(nextEntrypoint) && nextEntrypoint !== 'main') {
return nextEntrypoint;
}
return 'transform';
}
function resolvePythonRuntimeMode(value: unknown): PythonRuntimeMode {
if (value === 'main-thread') {
return 'main-thread';
}
return 'webworker';
}
async function getConfiguredPythonRuntimeMode(): Promise<PythonRuntimeMode> {
const metadata = await getMetaEngine().getProjectMetadata();
return resolvePythonRuntimeMode((metadata as { pythonRuntimeMode?: unknown } | null)?.pythonRuntimeMode);
}
class PythonBlogmarkTransformExecutor implements BlogmarkTransformExecutor { class PythonBlogmarkTransformExecutor implements BlogmarkTransformExecutor {
private runtimePromise: Promise<any> | null = null; private runtimePromise: Promise<any> | null = null;
@@ -169,7 +195,7 @@ def toast(message):
await runtime.runPythonAsync(script.content); await runtime.runPythonAsync(script.content);
const requestedEntrypoint = this.resolveEntrypoint(script.entrypoint); const requestedEntrypoint = resolveTransformEntrypoint(script.entrypoint);
const payload = JSON.stringify(input); const payload = JSON.stringify(input);
runtime.globals.set('__bds_transform_payload_json', payload); runtime.globals.set('__bds_transform_payload_json', payload);
runtime.globals.set('__bds_transform_entrypoint', requestedEntrypoint); runtime.globals.set('__bds_transform_entrypoint', requestedEntrypoint);
@@ -200,15 +226,6 @@ json.dumps(_result)
}; };
} }
private resolveEntrypoint(value: string): string {
const nextEntrypoint = typeof value === 'string' ? value.trim() : '';
if (/^[A-Za-z_][A-Za-z0-9_]*$/.test(nextEntrypoint) && nextEntrypoint !== 'main') {
return nextEntrypoint;
}
return 'transform';
}
private async getRuntime(): Promise<any> { private async getRuntime(): Promise<any> {
if (!this.runtimePromise) { if (!this.runtimePromise) {
this.runtimePromise = (async () => { this.runtimePromise = (async () => {
@@ -221,11 +238,26 @@ json.dumps(_result)
} }
} }
class PythonWorkerBlogmarkTransformExecutor implements BlogmarkTransformExecutor {
async runTransform(script: BlogmarkTransformScriptRecord, input: BlogmarkTransformInput): Promise<unknown> {
return getBlogmarkPythonWorkerRuntime().executeTransform({
scriptContent: script.content,
entrypoint: resolveTransformEntrypoint(script.entrypoint),
payloadJson: JSON.stringify(input),
});
}
}
const mainThreadExecutor = new PythonBlogmarkTransformExecutor();
const workerExecutor = new PythonWorkerBlogmarkTransformExecutor();
export class BlogmarkTransformService { export class BlogmarkTransformService {
constructor( constructor(
private readonly dependencies: { private readonly dependencies: {
provider?: BlogmarkTransformScriptProvider; provider?: BlogmarkTransformScriptProvider;
executor?: BlogmarkTransformExecutor; executor?: BlogmarkTransformExecutor;
resolvePythonRuntimeMode?: () => Promise<PythonRuntimeMode>;
executors?: Partial<Record<PythonRuntimeMode, BlogmarkTransformExecutor>>;
} = {}, } = {},
) {} ) {}
@@ -237,7 +269,7 @@ export class BlogmarkTransformService {
}; };
const provider = this.dependencies.provider ?? scriptEngineBackedProvider; const provider = this.dependencies.provider ?? scriptEngineBackedProvider;
const executor = this.dependencies.executor ?? new PythonBlogmarkTransformExecutor(); const executor = this.dependencies.executor ?? await this.resolveExecutorForConfiguredRuntime();
const scripts = await provider.getScripts(); const scripts = await provider.getScripts();
const activeTransforms = scripts const activeTransforms = scripts
@@ -303,6 +335,18 @@ export class BlogmarkTransformService {
toasts, toasts,
}; };
} }
private async resolveExecutorForConfiguredRuntime(): Promise<BlogmarkTransformExecutor> {
const resolveMode = this.dependencies.resolvePythonRuntimeMode ?? getConfiguredPythonRuntimeMode;
const mode = await resolveMode();
const executors = this.dependencies.executors ?? {};
if (mode === 'main-thread') {
return executors['main-thread'] ?? mainThreadExecutor;
}
return executors.webworker ?? workerExecutor;
}
} }
let blogmarkTransformServiceInstance: BlogmarkTransformService | null = null; let blogmarkTransformServiceInstance: BlogmarkTransformService | null = null;

View File

@@ -140,6 +140,14 @@ export interface GitPostFileChange {
previousPath?: string; previousPath?: string;
} }
export type GitScriptFileChangeStatus = 'added' | 'modified' | 'deleted' | 'renamed';
export interface GitScriptFileChange {
status: GitScriptFileChangeStatus;
path: string;
previousPath?: string;
}
type GitProvider = 'unknown' | 'github' | 'gitlab' | 'gitea-forgejo'; type GitProvider = 'unknown' | 'github' | 'gitlab' | 'gitea-forgejo';
let gitEngineInstance: GitEngine | null = null; let gitEngineInstance: GitEngine | null = null;
@@ -526,7 +534,12 @@ export class GitEngine {
return this.markdownExtensions.has(extension); return this.markdownExtensions.has(extension);
} }
private parseNameStatusOutput(raw: string): GitPostFileChange[] { private isScriptsPythonPath(value: string): boolean {
const normalized = this.normalizeRepoRelativePath(value);
return normalized.startsWith('scripts/') && path.extname(normalized).toLowerCase() === '.py';
}
private parseNameStatusOutput(raw: string, pathMatcher: (value: string) => boolean): GitPostFileChange[] {
const tokens = raw.split('\0').filter((token) => token.length > 0); const tokens = raw.split('\0').filter((token) => token.length > 0);
const changes: GitPostFileChange[] = []; const changes: GitPostFileChange[] = [];
@@ -543,7 +556,7 @@ export class GitEngine {
const previousPath = this.normalizeRepoRelativePath(previousPathRaw); const previousPath = this.normalizeRepoRelativePath(previousPathRaw);
const pathValue = this.normalizeRepoRelativePath(nextPathRaw); const pathValue = this.normalizeRepoRelativePath(nextPathRaw);
if (this.isPostsMarkdownPath(previousPath) || this.isPostsMarkdownPath(pathValue)) { if (pathMatcher(previousPath) || pathMatcher(pathValue)) {
changes.push({ changes.push({
status: 'renamed', status: 'renamed',
path: pathValue, path: pathValue,
@@ -555,7 +568,7 @@ export class GitEngine {
const filePathRaw = tokens[index++] ?? ''; const filePathRaw = tokens[index++] ?? '';
const filePath = this.normalizeRepoRelativePath(filePathRaw); const filePath = this.normalizeRepoRelativePath(filePathRaw);
if (!this.isPostsMarkdownPath(filePath)) { if (!pathMatcher(filePath)) {
continue; continue;
} }
@@ -1338,13 +1351,40 @@ export class GitEngine {
try { try {
const output = await git.raw(args); const output = await git.raw(args);
return this.parseNameStatusOutput(output); return this.parseNameStatusOutput(output, (value) => this.isPostsMarkdownPath(value));
} catch (error) { } catch (error) {
const message = error instanceof Error ? error.message : String(error ?? ''); const message = error instanceof Error ? error.message : String(error ?? '');
if (this.isSpawnBadFileDescriptorError(message)) { if (this.isSpawnBadFileDescriptorError(message)) {
try { try {
const output = await this.runGitCli(projectPath, args); const output = await this.runGitCli(projectPath, args);
return this.parseNameStatusOutput(output); return this.parseNameStatusOutput(output, (value) => this.isPostsMarkdownPath(value));
} catch {
return [];
}
}
return [];
}
}
async getChangedScriptFilesBetween(projectPath: string, fromCommit: string, toCommit: string): Promise<GitScriptFileChange[]> {
const fromRef = fromCommit.trim();
const toRef = toCommit.trim();
if (!fromRef || !toRef || fromRef === toRef) {
return [];
}
const git = this.createNonInteractiveGit(projectPath);
const args = ['diff', '--name-status', '--find-renames', '-z', `${fromRef}..${toRef}`, '--', 'scripts'];
try {
const output = await git.raw(args);
return this.parseNameStatusOutput(output, (value) => this.isScriptsPythonPath(value));
} catch (error) {
const message = error instanceof Error ? error.message : String(error ?? '');
if (this.isSpawnBadFileDescriptorError(message)) {
try {
const output = await this.runGitCli(projectPath, args);
return this.parseNameStatusOutput(output, (value) => this.isScriptsPythonPath(value));
} catch { } catch {
return []; return [];
} }

View File

@@ -24,6 +24,7 @@ export interface ProjectMetadata {
defaultAuthor?: string; // Default author for new posts and media defaultAuthor?: string; // Default author for new posts and media
maxPostsPerPage?: number; // Preview/server maximum posts per page (default 50) maxPostsPerPage?: number; // Preview/server maximum posts per page (default 50)
blogmarkCategory?: string; // Category used for externally captured bookmark posts blogmarkCategory?: string; // Category used for externally captured bookmark posts
pythonRuntimeMode?: 'webworker' | 'main-thread'; // Runtime mode for Python script execution
picoTheme?: PicoThemeName; // Selected Pico CSS theme for preview/rendering picoTheme?: PicoThemeName; // Selected Pico CSS theme for preview/rendering
categoryMetadata?: Record<string, CategoryMetadata>; // Per-category metadata for UI/rendering categoryMetadata?: Record<string, CategoryMetadata>; // Per-category metadata for UI/rendering
categorySettings?: Record<string, CategoryRenderSettings>; // Per-category list rendering preferences categorySettings?: Record<string, CategoryRenderSettings>; // Per-category list rendering preferences
@@ -85,6 +86,7 @@ function normalizeProjectMetadata(metadata: ProjectMetadata): ProjectMetadata {
const blogmarkCategory = typeof metadata.blogmarkCategory === 'string' const blogmarkCategory = typeof metadata.blogmarkCategory === 'string'
? normalizeNonEmptyTaxonomyTerm(metadata.blogmarkCategory) ?? undefined ? normalizeNonEmptyTaxonomyTerm(metadata.blogmarkCategory) ?? undefined
: undefined; : undefined;
const pythonRuntimeMode = metadata.pythonRuntimeMode === 'main-thread' ? 'main-thread' : 'webworker';
const picoTheme = sanitizePicoTheme(metadata.picoTheme); const picoTheme = sanitizePicoTheme(metadata.picoTheme);
const categoryMetadata = normalizeCategoryMetadata(metadata.categoryMetadata ?? metadata.categorySettings); const categoryMetadata = normalizeCategoryMetadata(metadata.categoryMetadata ?? metadata.categorySettings);
return { return {
@@ -92,6 +94,7 @@ function normalizeProjectMetadata(metadata: ProjectMetadata): ProjectMetadata {
publicUrl, publicUrl,
maxPostsPerPage, maxPostsPerPage,
blogmarkCategory, blogmarkCategory,
pythonRuntimeMode,
picoTheme, picoTheme,
categoryMetadata, categoryMetadata,
categorySettings: undefined, categorySettings: undefined,
@@ -306,6 +309,7 @@ export class MetaEngine extends EventEmitter {
defaultAuthor: normalizedUpdates.defaultAuthor, defaultAuthor: normalizedUpdates.defaultAuthor,
maxPostsPerPage: normalizedUpdates.maxPostsPerPage, maxPostsPerPage: normalizedUpdates.maxPostsPerPage,
blogmarkCategory: normalizedUpdates.blogmarkCategory, blogmarkCategory: normalizedUpdates.blogmarkCategory,
pythonRuntimeMode: normalizedUpdates.pythonRuntimeMode,
picoTheme: normalizedUpdates.picoTheme, picoTheme: normalizedUpdates.picoTheme,
categoryMetadata: normalizedUpdates.categoryMetadata, categoryMetadata: normalizedUpdates.categoryMetadata,
}); });

View File

@@ -42,6 +42,37 @@ export interface UpdateScriptInput {
enabled?: boolean; enabled?: boolean;
} }
export type GitScriptFileChangeStatus = 'added' | 'modified' | 'deleted' | 'renamed';
export interface GitScriptFileChange {
status: GitScriptFileChangeStatus;
path: string;
previousPath?: string;
}
export interface ScriptReconcileResult {
created: number;
updated: number;
deleted: number;
processedFiles: number;
}
interface ParsedScriptFile {
metadata: {
id?: string;
projectId?: string;
slug?: string;
title?: string;
kind?: string;
entrypoint?: string;
enabled?: boolean;
version?: number;
createdAt?: string;
updatedAt?: string;
};
body: string;
}
export class ScriptEngine extends EventEmitter { export class ScriptEngine extends EventEmitter {
private currentProjectId = 'default'; private currentProjectId = 'default';
private dataDir: string | null = null; private dataDir: string | null = null;
@@ -191,6 +222,205 @@ export class ScriptEngine extends EventEmitter {
return Promise.all(rows.map((item) => this.toScriptData(item))); return Promise.all(rows.map((item) => this.toScriptData(item)));
} }
async rebuildDatabaseFromFiles(): Promise<void> {
const db = getDatabase().getLocal();
const scriptsDir = this.getScriptsDir();
await db.delete(scripts).where(eq(scripts.projectId, this.currentProjectId));
const pythonFiles = await this.scanScriptFiles(scriptsDir);
if (pythonFiles.length === 0) {
this.emit('scriptsRebuilt');
return;
}
const usedIds = new Set<string>();
const insertedRows: Script[] = [];
for (const filePath of pythonFiles) {
const parsed = await this.readScriptFileWithMetadata(filePath);
if (!parsed) {
continue;
}
const desiredSlug = this.normalizeSlug(parsed.metadata.slug || path.basename(filePath, '.py'));
const slug = this.ensureUniqueSlug(desiredSlug, insertedRows);
const desiredId = typeof parsed.metadata.id === 'string' && parsed.metadata.id.trim().length > 0
? parsed.metadata.id.trim()
: uuidv4();
const id = usedIds.has(desiredId) ? uuidv4() : desiredId;
const now = new Date();
const row: NewScript = {
id,
projectId: this.currentProjectId,
slug,
title: this.normalizeTitle(parsed.metadata.title, slug),
kind: this.normalizeKind(parsed.metadata.kind),
entrypoint: this.normalizeEntrypoint(parsed.metadata.entrypoint),
enabled: this.normalizeEnabled(parsed.metadata.enabled),
version: this.normalizeVersion(parsed.metadata.version),
filePath,
createdAt: this.normalizeDate(parsed.metadata.createdAt, now),
updatedAt: this.normalizeDate(parsed.metadata.updatedAt, now),
};
await db.insert(scripts).values(row);
insertedRows.push(row as Script);
usedIds.add(id);
}
this.emit('scriptsRebuilt');
}
async reconcileScriptsFromGitChanges(projectPath: string, changes: GitScriptFileChange[]): Promise<ScriptReconcileResult> {
const db = getDatabase().getLocal();
const normalizedProjectPath = path.resolve(projectPath);
const relevantChanges = changes.filter((change) => {
if (!this.isPythonScriptPath(change.path)) {
return false;
}
if (change.status === 'renamed' && change.previousPath && !this.isPythonScriptPath(change.previousPath) && !this.isPythonScriptPath(change.path)) {
return false;
}
return true;
});
if (relevantChanges.length === 0) {
return { created: 0, updated: 0, deleted: 0, processedFiles: 0 };
}
const scriptRows = await this.getAllScriptRows();
const scriptsByPath = new Map<string, Script>();
for (const row of scriptRows) {
scriptsByPath.set(this.normalizePathForCompare(row.filePath), row);
}
let created = 0;
let updated = 0;
let deleted = 0;
let processedFiles = 0;
for (const change of relevantChanges) {
const absolutePath = this.normalizePathForCompare(path.resolve(normalizedProjectPath, change.path));
const previousAbsolutePath = change.previousPath
? this.normalizePathForCompare(path.resolve(normalizedProjectPath, change.previousPath))
: null;
if (change.status === 'deleted') {
const existing = scriptsByPath.get(absolutePath);
if (!existing) {
continue;
}
await db.delete(scripts).where(and(eq(scripts.id, existing.id), eq(scripts.projectId, this.currentProjectId)));
scriptsByPath.delete(absolutePath);
this.emit('scriptDeleted', existing.id);
deleted += 1;
processedFiles += 1;
continue;
}
let existing = previousAbsolutePath
? (scriptsByPath.get(previousAbsolutePath) || scriptsByPath.get(absolutePath))
: scriptsByPath.get(absolutePath);
const parsed = await this.readScriptFileWithMetadata(absolutePath);
if (!parsed) {
continue;
}
const allRows = await this.getAllScriptRows();
const parsedId = typeof parsed.metadata.id === 'string' ? parsed.metadata.id.trim() : '';
if (!existing && parsedId.length > 0) {
const byId = allRows.find((row) => row.id === parsedId);
if (byId) {
existing = byId;
}
}
const desiredSlug = this.normalizeSlug(parsed.metadata.slug || path.basename(absolutePath, '.py'));
const slug = this.ensureUniqueSlug(desiredSlug, allRows, existing?.id);
if (existing) {
const updateNow = new Date();
const nextRow = {
title: this.normalizeTitle(parsed.metadata.title, slug, existing.title),
slug,
kind: this.normalizeKind(parsed.metadata.kind, existing.kind),
entrypoint: this.normalizeEntrypoint(parsed.metadata.entrypoint, existing.entrypoint),
enabled: this.normalizeEnabled(parsed.metadata.enabled, existing.enabled),
version: this.normalizeVersion(parsed.metadata.version, existing.version),
filePath: absolutePath,
createdAt: this.normalizeDate(parsed.metadata.createdAt, existing.createdAt),
updatedAt: this.normalizeDate(parsed.metadata.updatedAt, updateNow),
};
await db.update(scripts)
.set(nextRow)
.where(and(eq(scripts.id, existing.id), eq(scripts.projectId, this.currentProjectId)));
const updatedRow = await this.getScriptRow(existing.id);
if (updatedRow) {
const updatedScript = await this.toScriptData(updatedRow);
this.emit('scriptUpdated', updatedScript);
}
if (previousAbsolutePath) {
scriptsByPath.delete(previousAbsolutePath);
}
scriptsByPath.set(absolutePath, {
...existing,
...nextRow,
});
updated += 1;
processedFiles += 1;
continue;
}
const desiredId = typeof parsed.metadata.id === 'string' && parsed.metadata.id.trim().length > 0
? parsed.metadata.id.trim()
: uuidv4();
const idExists = allRows.some((row) => row.id === desiredId);
const rowId = idExists ? uuidv4() : desiredId;
const now = new Date();
const newRow: NewScript = {
id: rowId,
projectId: this.currentProjectId,
slug,
title: this.normalizeTitle(parsed.metadata.title, slug),
kind: this.normalizeKind(parsed.metadata.kind),
entrypoint: this.normalizeEntrypoint(parsed.metadata.entrypoint),
enabled: this.normalizeEnabled(parsed.metadata.enabled),
version: this.normalizeVersion(parsed.metadata.version),
filePath: absolutePath,
createdAt: this.normalizeDate(parsed.metadata.createdAt, now),
updatedAt: this.normalizeDate(parsed.metadata.updatedAt, now),
};
await db.insert(scripts).values(newRow);
const createdRow = await this.getScriptRow(newRow.id);
if (createdRow) {
const createdScript = await this.toScriptData(createdRow);
this.emit('scriptCreated', createdScript);
}
scriptsByPath.set(absolutePath, newRow as Script);
created += 1;
processedFiles += 1;
}
return {
created,
updated,
deleted,
processedFiles,
};
}
private async getScriptRow(id: string): Promise<Script | null> { private async getScriptRow(id: string): Promise<Script | null> {
const rows = await this.getAllScriptRows(); const rows = await this.getAllScriptRows();
return rows.find((item) => item.id === id) || null; return rows.find((item) => item.id === id) || null;
@@ -240,6 +470,15 @@ export class ScriptEngine extends EventEmitter {
return path.join(this.getScriptsDir(), `${slug}.py`); return path.join(this.getScriptsDir(), `${slug}.py`);
} }
private normalizePathForCompare(filePath: string): string {
return path.resolve(filePath).replace(/\\/g, '/');
}
private isPythonScriptPath(value: string): boolean {
const normalized = value.replace(/\\/g, '/').replace(/^\.\//, '');
return normalized.startsWith('scripts/') && path.extname(normalized).toLowerCase() === '.py';
}
private normalizeSlug(value: string): string { private normalizeSlug(value: string): string {
const normalized = value const normalized = value
.toLowerCase() .toLowerCase()
@@ -306,6 +545,183 @@ export class ScriptEngine extends EventEmitter {
return rawContent.replace(frontmatterDocstringPattern, ''); return rawContent.replace(frontmatterDocstringPattern, '');
} }
private parseScriptFile(rawContent: string): ParsedScriptFile {
const frontmatterDocstringPattern = /^(?:"""|''')\r?\n---\r?\n([\s\S]*?)\r?\n---\r?\n(?:"""|''')\r?\n?/;
const match = rawContent.match(frontmatterDocstringPattern);
if (!match) {
return {
metadata: {},
body: rawContent,
};
}
const metadataLines = (match[1] || '').split(/\r?\n/);
const metadata: ParsedScriptFile['metadata'] = {};
for (const rawLine of metadataLines) {
const line = rawLine.trim();
if (!line || line.startsWith('#')) {
continue;
}
const separatorIndex = line.indexOf(':');
if (separatorIndex <= 0) {
continue;
}
const key = line.slice(0, separatorIndex).trim();
const valueRaw = line.slice(separatorIndex + 1).trim();
const value = this.parseYamlScalar(valueRaw);
if (key === 'enabled') {
if (typeof value === 'boolean') {
metadata.enabled = value;
}
continue;
}
if (key === 'version') {
const parsed = Number(value);
if (Number.isFinite(parsed)) {
metadata.version = parsed;
}
continue;
}
if (
key === 'id' ||
key === 'projectId' ||
key === 'slug' ||
key === 'title' ||
key === 'kind' ||
key === 'entrypoint' ||
key === 'createdAt' ||
key === 'updatedAt'
) {
if (typeof value === 'string') {
metadata[key] = value;
}
}
}
return {
metadata,
body: rawContent.replace(frontmatterDocstringPattern, ''),
};
}
private parseYamlScalar(valueRaw: string): string | number | boolean {
if ((valueRaw.startsWith('"') && valueRaw.endsWith('"')) || (valueRaw.startsWith("'") && valueRaw.endsWith("'"))) {
return valueRaw.slice(1, -1)
.replace(/\\"/g, '"')
.replace(/\\\\/g, '\\');
}
if (valueRaw === 'true') {
return true;
}
if (valueRaw === 'false') {
return false;
}
const numeric = Number(valueRaw);
if (!Number.isNaN(numeric)) {
return numeric;
}
return valueRaw;
}
private normalizeKind(kind: string | undefined, fallback: ScriptKind = 'utility'): ScriptKind {
if (kind === 'macro' || kind === 'utility' || kind === 'transform') {
return kind;
}
return fallback;
}
private normalizeEntrypoint(entrypoint: string | undefined, fallback = 'render'): string {
if (typeof entrypoint === 'string' && entrypoint.trim().length > 0) {
return entrypoint.trim();
}
return fallback;
}
private normalizeEnabled(enabled: boolean | undefined, fallback = true): boolean {
if (typeof enabled === 'boolean') {
return enabled;
}
return fallback;
}
private normalizeVersion(version: number | undefined, fallback = 1): number {
if (typeof version === 'number' && Number.isFinite(version) && version > 0) {
return Math.floor(version);
}
return fallback;
}
private normalizeDate(value: string | undefined, fallback: Date): Date {
if (typeof value === 'string') {
const parsed = new Date(value);
if (!Number.isNaN(parsed.getTime())) {
return parsed;
}
}
return fallback;
}
private normalizeTitle(title: string | undefined, slug: string, fallback?: string): string {
if (typeof title === 'string' && title.trim().length > 0) {
return title.trim();
}
if (typeof fallback === 'string' && fallback.trim().length > 0) {
return fallback.trim();
}
return slug;
}
private async scanScriptFiles(dir: string): Promise<string[]> {
const results: string[] = [];
const scan = async (currentDir: string): Promise<void> => {
let entries: Array<{ name: string; isDirectory(): boolean; isFile(): boolean }> = [];
try {
entries = await fs.readdir(currentDir, { withFileTypes: true }) as Array<{ name: string; isDirectory(): boolean; isFile(): boolean }>;
} catch {
return;
}
for (const entry of entries) {
const fullPath = path.join(currentDir, entry.name);
if (entry.isDirectory()) {
await scan(fullPath);
continue;
}
if (entry.isFile() && path.extname(entry.name).toLowerCase() === '.py') {
results.push(fullPath);
}
}
};
await scan(dir);
return results;
}
private async readScriptFileWithMetadata(filePath: string): Promise<ParsedScriptFile | null> {
try {
const rawContent = await fs.readFile(filePath, 'utf-8');
return this.parseScriptFile(rawContent);
} catch (error) {
const fsError = error as NodeJS.ErrnoException;
if (fsError.code !== 'ENOENT') {
throw error;
}
return null;
}
}
private async readScriptBody(filePath: string): Promise<string> { private async readScriptBody(filePath: string): Promise<string> {
try { try {
const rawContent = await fs.readFile(filePath, 'utf-8'); const rawContent = await fs.readFile(filePath, 'utf-8');

View File

@@ -0,0 +1,150 @@
import { parentPort } from 'worker_threads';
interface WorkerRunTransformRequest {
type: 'runTransform';
requestId: string;
scriptContent: string;
entrypoint: string;
payloadJson: string;
}
interface WorkerReadyMessage {
type: 'ready';
}
interface WorkerResultMessage {
type: 'transformResult';
requestId: string;
output: unknown;
toasts: string[];
}
interface WorkerErrorMessage {
type: 'transformError';
requestId: string;
error: string;
}
interface WorkerFatalErrorMessage {
type: 'error';
error: string;
}
type WorkerResponseMessage = WorkerReadyMessage | WorkerResultMessage | WorkerErrorMessage | WorkerFatalErrorMessage;
type PyodideRuntime = {
globals: {
set: (name: string, value: unknown) => void;
} | any;
runPythonAsync: (code: string) => Promise<unknown>;
};
const MAX_TOASTS_PER_SCRIPT = 5;
const MAX_TOAST_LENGTH = 300;
let runtimePromise: Promise<PyodideRuntime> | null = null;
function postMessage(message: WorkerResponseMessage): void {
parentPort?.postMessage(message);
}
function normalizeToastMessage(value: unknown): string | null {
if (value === undefined || value === null) {
return null;
}
const normalized = String(value).trim();
if (normalized.length === 0) {
return null;
}
return normalized.slice(0, MAX_TOAST_LENGTH);
}
async function getRuntime(): Promise<PyodideRuntime> {
if (!runtimePromise) {
runtimePromise = (async () => {
const pyodideModule = await import('pyodide');
return (await pyodideModule.loadPyodide()) as unknown as PyodideRuntime;
})();
}
return runtimePromise;
}
async function runTransform(request: WorkerRunTransformRequest): Promise<void> {
try {
const runtime = await getRuntime();
const toastMessages: string[] = [];
const pushToast = (message: unknown): void => {
if (toastMessages.length >= MAX_TOASTS_PER_SCRIPT) {
return;
}
const normalized = normalizeToastMessage(message);
if (!normalized) {
return;
}
toastMessages.push(normalized);
};
runtime.globals.set('__bds_push_toast', pushToast);
await runtime.runPythonAsync(`
def toast(message):
__bds_push_toast(str(message))
`);
await runtime.runPythonAsync(request.scriptContent);
runtime.globals.set('__bds_transform_payload_json', request.payloadJson);
runtime.globals.set('__bds_transform_entrypoint', request.entrypoint);
const rawResult = await runtime.runPythonAsync(`
import json
_payload = json.loads(__bds_transform_payload_json)
_entrypoint = __bds_transform_entrypoint
_transform_fn = globals().get(_entrypoint)
if _transform_fn is None or not callable(_transform_fn):
raise RuntimeError(f"Transform entrypoint '{_entrypoint}' is not callable")
_post = _payload.get("post")
if not isinstance(_post, dict):
raise RuntimeError("Transform payload is missing a valid 'post' object")
_context = _payload.get("context")
try:
_result = _transform_fn(_post, _context)
except TypeError:
_result = _transform_fn(_post)
if _result is None:
_result = _post
json.dumps(_result)
`);
postMessage({
type: 'transformResult',
requestId: request.requestId,
output: JSON.parse(String(rawResult)),
toasts: toastMessages,
});
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
postMessage({ type: 'transformError', requestId: request.requestId, error: message });
}
}
parentPort?.on('message', (message: WorkerRunTransformRequest) => {
if (message.type !== 'runTransform') {
return;
}
void runTransform(message);
});
void getRuntime()
.then(() => {
postMessage({ type: 'ready' });
})
.catch((error) => {
const message = error instanceof Error ? error.message : String(error);
postMessage({ type: 'error', error: message });
});

View File

@@ -185,8 +185,11 @@ export function registerIpcHandlers(): void {
return pullResult; return pullResult;
} }
const changedPostFiles = await engine.getChangedPostFilesBetween(projectPath, beforeHead, afterHead); const [changedPostFiles, changedScriptFiles] = await Promise.all([
if (changedPostFiles.length === 0) { engine.getChangedPostFilesBetween(projectPath, beforeHead, afterHead),
engine.getChangedScriptFilesBetween(projectPath, beforeHead, afterHead),
]);
if (changedPostFiles.length === 0 && changedScriptFiles.length === 0) {
return pullResult; return pullResult;
} }
@@ -194,15 +197,24 @@ export function registerIpcHandlers(): void {
const projectEngine = getProjectEngine(); const projectEngine = getProjectEngine();
const project = await projectEngine.getActiveProject(); const project = await projectEngine.getActiveProject();
const postEngine = getPostEngine(); const postEngine = getPostEngine();
const scriptEngine = getScriptEngine();
if (project) { if (project) {
const dataDir = projectEngine.getDataDir(project.id, project.dataPath); const dataDir = projectEngine.getDataDir(project.id, project.dataPath);
postEngine.setProjectContext(project.id, dataDir); postEngine.setProjectContext(project.id, dataDir);
scriptEngine.setProjectContext(project.id, dataDir);
} }
await postEngine.reconcilePublishedPostsFromGitChanges(projectPath, changedPostFiles); await Promise.all([
changedPostFiles.length > 0
? postEngine.reconcilePublishedPostsFromGitChanges(projectPath, changedPostFiles)
: Promise.resolve(),
changedScriptFiles.length > 0
? scriptEngine.reconcileScriptsFromGitChanges(projectPath, changedScriptFiles)
: Promise.resolve(),
]);
} catch (error) { } catch (error) {
console.error('Failed to reconcile published posts after git pull:', error); console.error('Failed to reconcile published posts/scripts after git pull:', error);
} }
return pullResult; return pullResult;
@@ -755,6 +767,18 @@ export function registerIpcHandlers(): void {
return engine.getAllScripts(); return engine.getAllScripts();
}); });
safeHandle('scripts:rebuildFromFiles', async () => {
const projectEngine = getProjectEngine();
const project = await projectEngine.getActiveProject();
const engine = getScriptEngine();
if (project) {
const dataDir = projectEngine.getDataDir(project.id, project.dataPath);
engine.setProjectContext(project.id, dataDir);
}
await engine.rebuildDatabaseFromFiles();
return true;
});
// ============ Task Handlers ============ // ============ Task Handlers ============
safeHandle('tasks:getAll', async () => { safeHandle('tasks:getAll', async () => {
@@ -1006,7 +1030,7 @@ export function registerIpcHandlers(): void {
return engine.getProjectMetadata(); return engine.getProjectMetadata();
}); });
safeHandle('meta:updateProjectMetadata', async (_, updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; picoTheme?: import('../shared/picoThemes').PicoThemeName; categoryMetadata?: Record<string, { renderInLists: boolean; showTitle: boolean; title: string }>; categorySettings?: Record<string, { renderInLists: boolean; showTitle: boolean }> }) => { safeHandle('meta:updateProjectMetadata', async (_, updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; blogmarkCategory?: string; pythonRuntimeMode?: 'webworker' | 'main-thread'; picoTheme?: import('../shared/picoThemes').PicoThemeName; categoryMetadata?: Record<string, { renderInLists: boolean; showTitle: boolean; title: string }>; categorySettings?: Record<string, { renderInLists: boolean; showTitle: boolean }> }) => {
const engine = getMetaEngine(); const engine = getMetaEngine();
await ensureMetaContext(engine); await ensureMetaContext(engine);
await engine.updateProjectMetadata(updates); await engine.updateProjectMetadata(updates);

View File

@@ -108,6 +108,7 @@ export const electronAPI: ElectronAPI = {
delete: (id: string) => ipcRenderer.invoke('scripts:delete', id), delete: (id: string) => ipcRenderer.invoke('scripts:delete', id),
get: (id: string) => ipcRenderer.invoke('scripts:get', id), get: (id: string) => ipcRenderer.invoke('scripts:get', id),
getAll: () => ipcRenderer.invoke('scripts:getAll'), getAll: () => ipcRenderer.invoke('scripts:getAll'),
rebuildFromFiles: () => ipcRenderer.invoke('scripts:rebuildFromFiles'),
}, },
// Post-Media Links // Post-Media Links
@@ -172,7 +173,7 @@ export const electronAPI: ElectronAPI = {
syncOnStartup: () => ipcRenderer.invoke('meta:syncOnStartup'), syncOnStartup: () => ipcRenderer.invoke('meta:syncOnStartup'),
getProjectMetadata: () => ipcRenderer.invoke('meta:getProjectMetadata'), getProjectMetadata: () => ipcRenderer.invoke('meta:getProjectMetadata'),
setProjectMetadata: (metadata: { name: string; description?: string }) => ipcRenderer.invoke('meta:setProjectMetadata', metadata), setProjectMetadata: (metadata: { name: string; description?: string }) => ipcRenderer.invoke('meta:setProjectMetadata', metadata),
updateProjectMetadata: (updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; blogmarkCategory?: string; picoTheme?: import('./shared/picoThemes').PicoThemeName; categoryMetadata?: Record<string, { renderInLists: boolean; showTitle: boolean; title: string }>; categorySettings?: Record<string, { renderInLists: boolean; showTitle: boolean }> }) => ipcRenderer.invoke('meta:updateProjectMetadata', updates), updateProjectMetadata: (updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; blogmarkCategory?: string; pythonRuntimeMode?: 'webworker' | 'main-thread'; picoTheme?: import('./shared/picoThemes').PicoThemeName; categoryMetadata?: Record<string, { renderInLists: boolean; showTitle: boolean; title: string }>; categorySettings?: Record<string, { renderInLists: boolean; showTitle: boolean }> }) => ipcRenderer.invoke('meta:updateProjectMetadata', updates),
}, },
// Tag Management (advanced tag operations) // Tag Management (advanced tag operations)

View File

@@ -43,6 +43,7 @@ export interface ProjectMetadata {
defaultAuthor?: string; defaultAuthor?: string;
maxPostsPerPage?: number; maxPostsPerPage?: number;
blogmarkCategory?: string; blogmarkCategory?: string;
pythonRuntimeMode?: 'webworker' | 'main-thread';
picoTheme?: import('./picoThemes').PicoThemeName; picoTheme?: import('./picoThemes').PicoThemeName;
categoryMetadata?: Record<string, CategoryMetadata>; categoryMetadata?: Record<string, CategoryMetadata>;
categorySettings?: Record<string, CategoryRenderSettings>; categorySettings?: Record<string, CategoryRenderSettings>;
@@ -565,6 +566,7 @@ export interface ElectronAPI {
delete: (id: string) => Promise<boolean>; delete: (id: string) => Promise<boolean>;
get: (id: string) => Promise<ScriptData | null>; get: (id: string) => Promise<ScriptData | null>;
getAll: () => Promise<ScriptData[]>; getAll: () => Promise<ScriptData[]>;
rebuildFromFiles: () => Promise<void>;
}; };
postMedia: { postMedia: {
link: (postId: string, mediaId: string) => Promise<MediaLinkData>; link: (postId: string, mediaId: string) => Promise<MediaLinkData>;
@@ -619,7 +621,7 @@ export interface ElectronAPI {
syncOnStartup: () => Promise<{ tags: string[]; categories: string[]; projectMetadata: ProjectMetadata | null }>; syncOnStartup: () => Promise<{ tags: string[]; categories: string[]; projectMetadata: ProjectMetadata | null }>;
getProjectMetadata: () => Promise<ProjectMetadata | null>; getProjectMetadata: () => Promise<ProjectMetadata | null>;
setProjectMetadata: (metadata: { name: string; description?: string }) => Promise<ProjectMetadata | null>; setProjectMetadata: (metadata: { name: string; description?: string }) => Promise<ProjectMetadata | null>;
updateProjectMetadata: (updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; blogmarkCategory?: string; picoTheme?: import('./picoThemes').PicoThemeName; categoryMetadata?: Record<string, CategoryMetadata>; categorySettings?: Record<string, CategoryRenderSettings> }) => Promise<ProjectMetadata | null>; updateProjectMetadata: (updates: { name?: string; description?: string; dataPath?: string; publicUrl?: string; mainLanguage?: string; defaultAuthor?: string; maxPostsPerPage?: number; blogmarkCategory?: string; pythonRuntimeMode?: 'webworker' | 'main-thread'; picoTheme?: import('./picoThemes').PicoThemeName; categoryMetadata?: Record<string, CategoryMetadata>; categorySettings?: Record<string, CategoryRenderSettings> }) => Promise<ProjectMetadata | null>;
}; };
tags: { tags: {
getAll: () => Promise<TagData[]>; getAll: () => Promise<TagData[]>;

View File

@@ -10,7 +10,7 @@ import {
import './SettingsView.css'; import './SettingsView.css';
// Export category IDs for sidebar navigation // Export category IDs for sidebar navigation
export type SettingsCategory = 'project' | 'editor' | 'content' | 'ai' | 'publishing' | 'data'; export type SettingsCategory = 'project' | 'editor' | 'content' | 'ai' | 'technology' | 'publishing' | 'data';
// Scroll to a settings section by category ID // Scroll to a settings section by category ID
export const scrollToSettingsSection = (category: SettingsCategory) => { export const scrollToSettingsSection = (category: SettingsCategory) => {
@@ -150,6 +150,7 @@ export const SettingsView: React.FC = () => {
const [projectDefaultAuthor, setProjectDefaultAuthor] = useState(''); const [projectDefaultAuthor, setProjectDefaultAuthor] = useState('');
const [projectMaxPostsPerPage, setProjectMaxPostsPerPage] = useState(50); const [projectMaxPostsPerPage, setProjectMaxPostsPerPage] = useState(50);
const [projectBlogmarkCategory, setProjectBlogmarkCategory] = useState('article'); const [projectBlogmarkCategory, setProjectBlogmarkCategory] = useState('article');
const [projectPythonRuntimeMode, setProjectPythonRuntimeMode] = useState<'webworker' | 'main-thread'>('webworker');
// Post categories management // Post categories management
const [postCategories, setPostCategories] = useState<string[]>(DEFAULT_POST_CATEGORIES); const [postCategories, setPostCategories] = useState<string[]>(DEFAULT_POST_CATEGORIES);
@@ -208,6 +209,9 @@ export const SettingsView: React.FC = () => {
const incomingBlogmarkCategory = normalizeBlogmarkCategory((metadata as { blogmarkCategory?: unknown } | null)?.blogmarkCategory); const incomingBlogmarkCategory = normalizeBlogmarkCategory((metadata as { blogmarkCategory?: unknown } | null)?.blogmarkCategory);
setProjectBlogmarkCategory(incomingBlogmarkCategory || 'article'); setProjectBlogmarkCategory(incomingBlogmarkCategory || 'article');
const incomingPythonRuntimeMode = (metadata as { pythonRuntimeMode?: unknown } | null)?.pythonRuntimeMode;
setProjectPythonRuntimeMode(incomingPythonRuntimeMode === 'main-thread' ? 'main-thread' : 'webworker');
const incomingCategoryMetadata = (metadata as any)?.categoryMetadata as Record<string, CategoryMetadata> | undefined; const incomingCategoryMetadata = (metadata as any)?.categoryMetadata as Record<string, CategoryMetadata> | undefined;
const incomingLegacyCategorySettings = (metadata as any)?.categorySettings as Record<string, { renderInLists: boolean; showTitle: boolean }> | undefined; const incomingLegacyCategorySettings = (metadata as any)?.categorySettings as Record<string, { renderInLists: boolean; showTitle: boolean }> | undefined;
setCategoryMetadata((current) => { setCategoryMetadata((current) => {
@@ -342,6 +346,7 @@ export const SettingsView: React.FC = () => {
defaultAuthor: projectDefaultAuthor.trim() || undefined, defaultAuthor: projectDefaultAuthor.trim() || undefined,
maxPostsPerPage: Math.min(500, Math.max(1, Math.floor(projectMaxPostsPerPage || 50))), maxPostsPerPage: Math.min(500, Math.max(1, Math.floor(projectMaxPostsPerPage || 50))),
blogmarkCategory: normalizeBlogmarkCategory(projectBlogmarkCategory) || undefined, blogmarkCategory: normalizeBlogmarkCategory(projectBlogmarkCategory) || undefined,
pythonRuntimeMode: projectPythonRuntimeMode,
categoryMetadata, categoryMetadata,
}); });
} }
@@ -389,8 +394,9 @@ export const SettingsView: React.FC = () => {
const editorKeywords = ['editor', 'mode', 'wysiwyg', 'markdown', 'preview', 'visual']; const editorKeywords = ['editor', 'mode', 'wysiwyg', 'markdown', 'preview', 'visual'];
const contentKeywords = ['content', 'categories', 'post', 'article', 'picture', 'aside', 'page']; const contentKeywords = ['content', 'categories', 'post', 'article', 'picture', 'aside', 'page'];
const aiKeywords = ['ai', 'assistant', 'chat', 'model', 'prompt', 'system', 'api', 'key', 'claude', 'gpt', 'opencode']; const aiKeywords = ['ai', 'assistant', 'chat', 'model', 'prompt', 'system', 'api', 'key', 'claude', 'gpt', 'opencode'];
const technologyKeywords = ['technology', 'python', 'runtime', 'worker', 'webworker', 'main thread', 'execution'];
const publishingKeywords = ['publishing', 'ftp', 'ssh', 'deploy', 'server', 'host', 'upload']; const publishingKeywords = ['publishing', 'ftp', 'ssh', 'deploy', 'server', 'host', 'upload'];
const dataKeywords = ['data', 'database', 'rebuild', 'maintenance', 'posts', 'media', 'links', 'folder', 'filesystem']; const dataKeywords = ['data', 'database', 'rebuild', 'maintenance', 'posts', 'media', 'scripts', 'links', 'folder', 'filesystem'];
const renderProjectSettings = () => ( const renderProjectSettings = () => (
<SettingSection <SettingSection
@@ -1023,6 +1029,30 @@ export const SettingsView: React.FC = () => {
</SettingSection> </SettingSection>
); );
const renderTechnologySettings = () => (
<SettingSection
id="settings-section-technology"
title={t('settings.technology.title')}
description={t('settings.technology.description')}
hidden={!sectionHasMatches(technologyKeywords)}
>
<SettingRow
id="project-python-runtime-mode"
label={t('settings.technology.pythonRuntimeModeLabel')}
description={t('settings.technology.pythonRuntimeModeDescription')}
>
<select
id="project-python-runtime-mode"
value={projectPythonRuntimeMode}
onChange={(event) => setProjectPythonRuntimeMode(event.target.value as 'webworker' | 'main-thread')}
>
<option value="webworker">{t('settings.technology.pythonRuntimeMode.webworker')}</option>
<option value="main-thread">{t('settings.technology.pythonRuntimeMode.mainThread')}</option>
</select>
</SettingRow>
</SettingSection>
);
const renderPublishingSettings = () => ( const renderPublishingSettings = () => (
<> <>
<SettingSection <SettingSection
@@ -1205,6 +1235,29 @@ export const SettingsView: React.FC = () => {
</button> </button>
</SettingRow> </SettingRow>
<SettingRow
id="rebuild-scripts"
label={t('settings.data.rebuildScriptsLabel')}
description={t('settings.data.rebuildScriptsDescription')}
>
<button
className="secondary"
onClick={async () => {
showToast.loading(t('settings.toast.rebuildScriptsLoading'));
try {
await window.electronAPI?.scripts.rebuildFromFiles();
showToast.dismiss();
showToast.success(t('settings.toast.rebuildScriptsSuccess'));
} catch {
showToast.dismiss();
showToast.error(t('settings.toast.rebuildScriptsFailed'));
}
}}
>
{t('settings.data.rebuildScriptsAction')}
</button>
</SettingRow>
<SettingRow <SettingRow
id="rebuild-links" id="rebuild-links"
label={t('settings.data.rebuildLinksLabel')} label={t('settings.data.rebuildLinksLabel')}
@@ -1290,6 +1343,7 @@ export const SettingsView: React.FC = () => {
sectionHasMatches(editorKeywords) || sectionHasMatches(editorKeywords) ||
sectionHasMatches(contentKeywords) || sectionHasMatches(contentKeywords) ||
sectionHasMatches(aiKeywords) || sectionHasMatches(aiKeywords) ||
sectionHasMatches(technologyKeywords) ||
sectionHasMatches(publishingKeywords) || sectionHasMatches(publishingKeywords) ||
sectionHasMatches(dataKeywords); sectionHasMatches(dataKeywords);
@@ -1325,6 +1379,7 @@ export const SettingsView: React.FC = () => {
{renderEditorSettings()} {renderEditorSettings()}
{renderContentSettings()} {renderContentSettings()}
{renderAISettings()} {renderAISettings()}
{renderTechnologySettings()}
{renderPublishingSettings()} {renderPublishingSettings()}
{renderDataSettings()} {renderDataSettings()}
</> </>

View File

@@ -1261,7 +1261,7 @@ const SettingsNav: React.FC = () => {
const { tabs, activeTabId, openTab } = useAppStore(); const { tabs, activeTabId, openTab } = useAppStore();
const [activeSection, setActiveSection] = useState<SettingsCategory | null>(() => { const [activeSection, setActiveSection] = useState<SettingsCategory | null>(() => {
const persisted = getPersistedSidebarSection('settings'); const persisted = getPersistedSidebarSection('settings');
if (persisted === 'project' || persisted === 'editor' || persisted === 'content' || persisted === 'ai' || persisted === 'publishing' || persisted === 'data') { if (persisted === 'project' || persisted === 'editor' || persisted === 'content' || persisted === 'ai' || persisted === 'technology' || persisted === 'publishing' || persisted === 'data') {
return persisted; return persisted;
} }
return null; return null;
@@ -1322,6 +1322,13 @@ const SettingsNav: React.FC = () => {
<span className="settings-nav-entry-icon">🤖</span> <span className="settings-nav-entry-icon">🤖</span>
<span>{t('sidebar.nav.ai')}</span> <span>{t('sidebar.nav.ai')}</span>
</button> </button>
<button
className={`settings-nav-entry ${activeSection === 'technology' ? 'active' : ''}`}
onClick={() => handleNavClick('technology')}
>
<span className="settings-nav-entry-icon"></span>
<span>{t('sidebar.nav.technology')}</span>
</button>
<button <button
className={`settings-nav-entry ${activeSection === 'publishing' ? 'active' : ''}`} className={`settings-nav-entry ${activeSection === 'publishing' ? 'active' : ''}`}
onClick={() => handleNavClick('publishing')} onClick={() => handleNavClick('publishing')}

View File

@@ -127,6 +127,12 @@
"settings.content.showTitles": "Titel anzeigen", "settings.content.showTitles": "Titel anzeigen",
"settings.ai.title": "KI-Assistent", "settings.ai.title": "KI-Assistent",
"settings.ai.noModels": "Keine Modelle verfügbar", "settings.ai.noModels": "Keine Modelle verfügbar",
"settings.technology.title": "Technologie",
"settings.technology.description": "Konfiguriere das Laufzeitverhalten für die Python-Skriptausführung.",
"settings.technology.pythonRuntimeModeLabel": "Python-Laufzeitmodus",
"settings.technology.pythonRuntimeModeDescription": "Lege fest, wo Python-Skripte für Transformationspipelines ausgeführt werden.",
"settings.technology.pythonRuntimeMode.webworker": "Web Worker (empfohlen)",
"settings.technology.pythonRuntimeMode.mainThread": "Hauptthread (Legacy)",
"settings.publishing.ftpTitle": "FTP-Veröffentlichung", "settings.publishing.ftpTitle": "FTP-Veröffentlichung",
"settings.publishing.sshTitle": "SSH-Veröffentlichung", "settings.publishing.sshTitle": "SSH-Veröffentlichung",
"settings.data.title": "Datenbankwartung", "settings.data.title": "Datenbankwartung",
@@ -167,6 +173,9 @@
"settings.toast.rebuildMediaLoading": "Mediendatenbank wird neu aufgebaut...", "settings.toast.rebuildMediaLoading": "Mediendatenbank wird neu aufgebaut...",
"settings.toast.rebuildMediaSuccess": "Mediendatenbank neu aufgebaut", "settings.toast.rebuildMediaSuccess": "Mediendatenbank neu aufgebaut",
"settings.toast.rebuildMediaFailed": "Mediendatenbank konnte nicht neu aufgebaut werden", "settings.toast.rebuildMediaFailed": "Mediendatenbank konnte nicht neu aufgebaut werden",
"settings.toast.rebuildScriptsLoading": "Skriptdatenbank wird neu aufgebaut...",
"settings.toast.rebuildScriptsSuccess": "Skriptdatenbank neu aufgebaut",
"settings.toast.rebuildScriptsFailed": "Skriptdatenbank konnte nicht neu aufgebaut werden",
"settings.toast.rebuildLinksLoading": "Beitragslinks werden neu aufgebaut...", "settings.toast.rebuildLinksLoading": "Beitragslinks werden neu aufgebaut...",
"settings.toast.rebuildLinksSuccess": "Beitragslinks neu aufgebaut", "settings.toast.rebuildLinksSuccess": "Beitragslinks neu aufgebaut",
"settings.toast.rebuildLinksFailed": "Beitragslinks konnten nicht neu aufgebaut werden", "settings.toast.rebuildLinksFailed": "Beitragslinks konnten nicht neu aufgebaut werden",
@@ -421,6 +430,7 @@
"sidebar.nav.editor": "Texteditor", "sidebar.nav.editor": "Texteditor",
"sidebar.nav.content": "Inhalt", "sidebar.nav.content": "Inhalt",
"sidebar.nav.ai": "KI-Assistent", "sidebar.nav.ai": "KI-Assistent",
"sidebar.nav.technology": "Technologie",
"sidebar.nav.publishing": "Veröffentlichung", "sidebar.nav.publishing": "Veröffentlichung",
"sidebar.nav.data": "Daten", "sidebar.nav.data": "Daten",
"sidebar.nav.style": "Stil", "sidebar.nav.style": "Stil",
@@ -702,6 +712,9 @@
"settings.data.rebuildMediaLabel": "Mediendatenbank neu aufbauen", "settings.data.rebuildMediaLabel": "Mediendatenbank neu aufbauen",
"settings.data.rebuildMediaDescription": "Alle Mediendateien und Sidecar-Metadaten neu scannen. Fehlende Einträge werden neu erzeugt.", "settings.data.rebuildMediaDescription": "Alle Mediendateien und Sidecar-Metadaten neu scannen. Fehlende Einträge werden neu erzeugt.",
"settings.data.rebuildMediaAction": "Medien neu aufbauen", "settings.data.rebuildMediaAction": "Medien neu aufbauen",
"settings.data.rebuildScriptsLabel": "Skriptdatenbank neu aufbauen",
"settings.data.rebuildScriptsDescription": "Alle Python-Skripte neu scannen und den Skript-Metadatenindex neu aufbauen.",
"settings.data.rebuildScriptsAction": "Skripte neu aufbauen",
"settings.data.rebuildLinksLabel": "Beitragslinks neu aufbauen", "settings.data.rebuildLinksLabel": "Beitragslinks neu aufbauen",
"settings.data.rebuildLinksDescription": "Alle Beiträge neu scannen und den internen Linkgraphen zwischen Beiträgen neu aufbauen.", "settings.data.rebuildLinksDescription": "Alle Beiträge neu scannen und den internen Linkgraphen zwischen Beiträgen neu aufbauen.",
"settings.data.rebuildLinksAction": "Links neu aufbauen", "settings.data.rebuildLinksAction": "Links neu aufbauen",

View File

@@ -127,6 +127,12 @@
"settings.content.showTitles": "Show titles", "settings.content.showTitles": "Show titles",
"settings.ai.title": "AI Assistant", "settings.ai.title": "AI Assistant",
"settings.ai.noModels": "No models available", "settings.ai.noModels": "No models available",
"settings.technology.title": "Technology",
"settings.technology.description": "Configure runtime behavior for Python script execution.",
"settings.technology.pythonRuntimeModeLabel": "Python Runtime Mode",
"settings.technology.pythonRuntimeModeDescription": "Choose where Python scripts execute for transform pipelines.",
"settings.technology.pythonRuntimeMode.webworker": "Web Worker (Recommended)",
"settings.technology.pythonRuntimeMode.mainThread": "Main Thread (Legacy)",
"settings.publishing.ftpTitle": "FTP Publishing", "settings.publishing.ftpTitle": "FTP Publishing",
"settings.publishing.sshTitle": "SSH Publishing", "settings.publishing.sshTitle": "SSH Publishing",
"settings.data.title": "Database Maintenance", "settings.data.title": "Database Maintenance",
@@ -167,6 +173,9 @@
"settings.toast.rebuildMediaLoading": "Rebuilding media database...", "settings.toast.rebuildMediaLoading": "Rebuilding media database...",
"settings.toast.rebuildMediaSuccess": "Media database rebuilt", "settings.toast.rebuildMediaSuccess": "Media database rebuilt",
"settings.toast.rebuildMediaFailed": "Failed to rebuild media database", "settings.toast.rebuildMediaFailed": "Failed to rebuild media database",
"settings.toast.rebuildScriptsLoading": "Rebuilding scripts database...",
"settings.toast.rebuildScriptsSuccess": "Scripts database rebuilt",
"settings.toast.rebuildScriptsFailed": "Failed to rebuild scripts database",
"settings.toast.rebuildLinksLoading": "Rebuilding post links...", "settings.toast.rebuildLinksLoading": "Rebuilding post links...",
"settings.toast.rebuildLinksSuccess": "Post links rebuilt", "settings.toast.rebuildLinksSuccess": "Post links rebuilt",
"settings.toast.rebuildLinksFailed": "Failed to rebuild post links", "settings.toast.rebuildLinksFailed": "Failed to rebuild post links",
@@ -421,6 +430,7 @@
"sidebar.nav.editor": "Editor", "sidebar.nav.editor": "Editor",
"sidebar.nav.content": "Content", "sidebar.nav.content": "Content",
"sidebar.nav.ai": "AI Assistant", "sidebar.nav.ai": "AI Assistant",
"sidebar.nav.technology": "Technology",
"sidebar.nav.publishing": "Publishing", "sidebar.nav.publishing": "Publishing",
"sidebar.nav.data": "Data", "sidebar.nav.data": "Data",
"sidebar.nav.style": "Style", "sidebar.nav.style": "Style",
@@ -702,6 +712,9 @@
"settings.data.rebuildMediaLabel": "Rebuild Media Database", "settings.data.rebuildMediaLabel": "Rebuild Media Database",
"settings.data.rebuildMediaDescription": "Re-scan all media files and sidecar metadata. Regenerates missing entries.", "settings.data.rebuildMediaDescription": "Re-scan all media files and sidecar metadata. Regenerates missing entries.",
"settings.data.rebuildMediaAction": "Rebuild Media", "settings.data.rebuildMediaAction": "Rebuild Media",
"settings.data.rebuildScriptsLabel": "Rebuild Scripts Database",
"settings.data.rebuildScriptsDescription": "Re-scan all Python scripts and rebuild the scripts metadata index.",
"settings.data.rebuildScriptsAction": "Rebuild Scripts",
"settings.data.rebuildLinksLabel": "Rebuild Post Links", "settings.data.rebuildLinksLabel": "Rebuild Post Links",
"settings.data.rebuildLinksDescription": "Re-scan all posts and rebuild the internal link graph between posts.", "settings.data.rebuildLinksDescription": "Re-scan all posts and rebuild the internal link graph between posts.",
"settings.data.rebuildLinksAction": "Rebuild Links", "settings.data.rebuildLinksAction": "Rebuild Links",

View File

@@ -127,6 +127,12 @@
"settings.content.showTitles": "Mostrar títulos", "settings.content.showTitles": "Mostrar títulos",
"settings.ai.title": "Asistente IA", "settings.ai.title": "Asistente IA",
"settings.ai.noModels": "No hay modelos disponibles", "settings.ai.noModels": "No hay modelos disponibles",
"settings.technology.title": "Tecnología",
"settings.technology.description": "Configura el comportamiento de ejecución para scripts de Python.",
"settings.technology.pythonRuntimeModeLabel": "Modo de ejecución de Python",
"settings.technology.pythonRuntimeModeDescription": "Elige dónde se ejecutan los scripts de Python para los flujos de transformación.",
"settings.technology.pythonRuntimeMode.webworker": "Web Worker (recomendado)",
"settings.technology.pythonRuntimeMode.mainThread": "Hilo principal (heredado)",
"settings.publishing.ftpTitle": "Publicación FTP", "settings.publishing.ftpTitle": "Publicación FTP",
"settings.publishing.sshTitle": "Publicación SSH", "settings.publishing.sshTitle": "Publicación SSH",
"settings.data.title": "Mantenimiento de base de datos", "settings.data.title": "Mantenimiento de base de datos",
@@ -167,6 +173,9 @@
"settings.toast.rebuildMediaLoading": "Reconstruyendo base de datos de medios...", "settings.toast.rebuildMediaLoading": "Reconstruyendo base de datos de medios...",
"settings.toast.rebuildMediaSuccess": "Base de datos de medios reconstruida", "settings.toast.rebuildMediaSuccess": "Base de datos de medios reconstruida",
"settings.toast.rebuildMediaFailed": "No se pudo reconstruir la base de datos de medios", "settings.toast.rebuildMediaFailed": "No se pudo reconstruir la base de datos de medios",
"settings.toast.rebuildScriptsLoading": "Reconstruyendo base de datos de scripts...",
"settings.toast.rebuildScriptsSuccess": "Base de datos de scripts reconstruida",
"settings.toast.rebuildScriptsFailed": "No se pudo reconstruir la base de datos de scripts",
"settings.toast.rebuildLinksLoading": "Reconstruyendo enlaces de entradas...", "settings.toast.rebuildLinksLoading": "Reconstruyendo enlaces de entradas...",
"settings.toast.rebuildLinksSuccess": "Enlaces de publicaciones reconstruidos", "settings.toast.rebuildLinksSuccess": "Enlaces de publicaciones reconstruidos",
"settings.toast.rebuildLinksFailed": "No se pudieron reconstruir los enlaces de entradas", "settings.toast.rebuildLinksFailed": "No se pudieron reconstruir los enlaces de entradas",
@@ -421,6 +430,7 @@
"sidebar.nav.editor": "Editor", "sidebar.nav.editor": "Editor",
"sidebar.nav.content": "Contenido", "sidebar.nav.content": "Contenido",
"sidebar.nav.ai": "Asistente IA", "sidebar.nav.ai": "Asistente IA",
"sidebar.nav.technology": "Tecnología",
"sidebar.nav.publishing": "Publicación", "sidebar.nav.publishing": "Publicación",
"sidebar.nav.data": "Datos", "sidebar.nav.data": "Datos",
"sidebar.nav.style": "Estilo", "sidebar.nav.style": "Estilo",
@@ -702,6 +712,9 @@
"settings.data.rebuildMediaLabel": "Reconstruir base de datos de medios", "settings.data.rebuildMediaLabel": "Reconstruir base de datos de medios",
"settings.data.rebuildMediaDescription": "Reescanea todos los archivos multimedia y metadatos sidecar. Regenera las entradas faltantes.", "settings.data.rebuildMediaDescription": "Reescanea todos los archivos multimedia y metadatos sidecar. Regenera las entradas faltantes.",
"settings.data.rebuildMediaAction": "Reconstruir medios", "settings.data.rebuildMediaAction": "Reconstruir medios",
"settings.data.rebuildScriptsLabel": "Reconstruir base de datos de scripts",
"settings.data.rebuildScriptsDescription": "Reescanea todos los scripts de Python y reconstruye el índice de metadatos de scripts.",
"settings.data.rebuildScriptsAction": "Reconstruir scripts",
"settings.data.rebuildLinksLabel": "Reconstruir enlaces de publicaciones", "settings.data.rebuildLinksLabel": "Reconstruir enlaces de publicaciones",
"settings.data.rebuildLinksDescription": "Reescanea todas las publicaciones y reconstruye el grafo interno de enlaces entre publicaciones.", "settings.data.rebuildLinksDescription": "Reescanea todas las publicaciones y reconstruye el grafo interno de enlaces entre publicaciones.",
"settings.data.rebuildLinksAction": "Reconstruir enlaces", "settings.data.rebuildLinksAction": "Reconstruir enlaces",

View File

@@ -127,6 +127,12 @@
"settings.content.showTitles": "Afficher les titres", "settings.content.showTitles": "Afficher les titres",
"settings.ai.title": "Assistant IA", "settings.ai.title": "Assistant IA",
"settings.ai.noModels": "Aucun modèle disponible", "settings.ai.noModels": "Aucun modèle disponible",
"settings.technology.title": "Technologie",
"settings.technology.description": "Configurez le comportement dexécution des scripts Python.",
"settings.technology.pythonRuntimeModeLabel": "Mode dexécution Python",
"settings.technology.pythonRuntimeModeDescription": "Choisissez où les scripts Python sexécutent pour les pipelines de transformation.",
"settings.technology.pythonRuntimeMode.webworker": "Web Worker (recommandé)",
"settings.technology.pythonRuntimeMode.mainThread": "Thread principal (hérité)",
"settings.publishing.ftpTitle": "Publication FTP", "settings.publishing.ftpTitle": "Publication FTP",
"settings.publishing.sshTitle": "Publication SSH", "settings.publishing.sshTitle": "Publication SSH",
"settings.data.title": "Maintenance de la base de données", "settings.data.title": "Maintenance de la base de données",
@@ -167,6 +173,9 @@
"settings.toast.rebuildMediaLoading": "Reconstruction de la base des médias...", "settings.toast.rebuildMediaLoading": "Reconstruction de la base des médias...",
"settings.toast.rebuildMediaSuccess": "Base médias reconstruite", "settings.toast.rebuildMediaSuccess": "Base médias reconstruite",
"settings.toast.rebuildMediaFailed": "Impossible de reconstruire la base des médias", "settings.toast.rebuildMediaFailed": "Impossible de reconstruire la base des médias",
"settings.toast.rebuildScriptsLoading": "Reconstruction de la base des scripts...",
"settings.toast.rebuildScriptsSuccess": "Base des scripts reconstruite",
"settings.toast.rebuildScriptsFailed": "Impossible de reconstruire la base des scripts",
"settings.toast.rebuildLinksLoading": "Reconstruction des liens darticles...", "settings.toast.rebuildLinksLoading": "Reconstruction des liens darticles...",
"settings.toast.rebuildLinksSuccess": "Liens darticles reconstruits", "settings.toast.rebuildLinksSuccess": "Liens darticles reconstruits",
"settings.toast.rebuildLinksFailed": "Impossible de reconstruire les liens darticles", "settings.toast.rebuildLinksFailed": "Impossible de reconstruire les liens darticles",
@@ -421,6 +430,7 @@
"sidebar.nav.editor": "Éditeur", "sidebar.nav.editor": "Éditeur",
"sidebar.nav.content": "Contenu", "sidebar.nav.content": "Contenu",
"sidebar.nav.ai": "Assistant IA", "sidebar.nav.ai": "Assistant IA",
"sidebar.nav.technology": "Technologie",
"sidebar.nav.publishing": "Publication", "sidebar.nav.publishing": "Publication",
"sidebar.nav.data": "Données", "sidebar.nav.data": "Données",
"sidebar.nav.style": "Style", "sidebar.nav.style": "Style",
@@ -702,6 +712,9 @@
"settings.data.rebuildMediaLabel": "Reconstruire la base médias", "settings.data.rebuildMediaLabel": "Reconstruire la base médias",
"settings.data.rebuildMediaDescription": "Réanalyse tous les fichiers médias et leurs métadonnées sidecar. Régénère les entrées manquantes.", "settings.data.rebuildMediaDescription": "Réanalyse tous les fichiers médias et leurs métadonnées sidecar. Régénère les entrées manquantes.",
"settings.data.rebuildMediaAction": "Reconstruire les médias", "settings.data.rebuildMediaAction": "Reconstruire les médias",
"settings.data.rebuildScriptsLabel": "Reconstruire la base des scripts",
"settings.data.rebuildScriptsDescription": "Réanalyse tous les scripts Python et reconstruit lindex des métadonnées de scripts.",
"settings.data.rebuildScriptsAction": "Reconstruire les scripts",
"settings.data.rebuildLinksLabel": "Reconstruire les liens darticles", "settings.data.rebuildLinksLabel": "Reconstruire les liens darticles",
"settings.data.rebuildLinksDescription": "Réanalyse tous les articles et reconstruit le graphe interne des liens entre articles.", "settings.data.rebuildLinksDescription": "Réanalyse tous les articles et reconstruit le graphe interne des liens entre articles.",
"settings.data.rebuildLinksAction": "Reconstruire les liens", "settings.data.rebuildLinksAction": "Reconstruire les liens",

View File

@@ -127,6 +127,12 @@
"settings.content.showTitles": "Mostra titoli", "settings.content.showTitles": "Mostra titoli",
"settings.ai.title": "Assistente IA", "settings.ai.title": "Assistente IA",
"settings.ai.noModels": "Nessun modello disponibile", "settings.ai.noModels": "Nessun modello disponibile",
"settings.technology.title": "Tecnologia",
"settings.technology.description": "Configura il comportamento di runtime per l'esecuzione degli script Python.",
"settings.technology.pythonRuntimeModeLabel": "Modalità runtime Python",
"settings.technology.pythonRuntimeModeDescription": "Scegli dove eseguire gli script Python per le pipeline di trasformazione.",
"settings.technology.pythonRuntimeMode.webworker": "Web Worker (consigliato)",
"settings.technology.pythonRuntimeMode.mainThread": "Thread principale (legacy)",
"settings.publishing.ftpTitle": "Pubblicazione FTP", "settings.publishing.ftpTitle": "Pubblicazione FTP",
"settings.publishing.sshTitle": "Pubblicazione SSH", "settings.publishing.sshTitle": "Pubblicazione SSH",
"settings.data.title": "Manutenzione database", "settings.data.title": "Manutenzione database",
@@ -167,6 +173,9 @@
"settings.toast.rebuildMediaLoading": "Ricostruzione database media...", "settings.toast.rebuildMediaLoading": "Ricostruzione database media...",
"settings.toast.rebuildMediaSuccess": "Database media ricostruito", "settings.toast.rebuildMediaSuccess": "Database media ricostruito",
"settings.toast.rebuildMediaFailed": "Impossibile ricostruire il database dei media", "settings.toast.rebuildMediaFailed": "Impossibile ricostruire il database dei media",
"settings.toast.rebuildScriptsLoading": "Ricostruzione database script...",
"settings.toast.rebuildScriptsSuccess": "Database script ricostruito",
"settings.toast.rebuildScriptsFailed": "Impossibile ricostruire il database degli script",
"settings.toast.rebuildLinksLoading": "Ricostruzione dei link dei post...", "settings.toast.rebuildLinksLoading": "Ricostruzione dei link dei post...",
"settings.toast.rebuildLinksSuccess": "Link dei post ricostruiti", "settings.toast.rebuildLinksSuccess": "Link dei post ricostruiti",
"settings.toast.rebuildLinksFailed": "Impossibile ricostruire i link dei post", "settings.toast.rebuildLinksFailed": "Impossibile ricostruire i link dei post",
@@ -421,6 +430,7 @@
"sidebar.nav.editor": "Editor", "sidebar.nav.editor": "Editor",
"sidebar.nav.content": "Contenuto", "sidebar.nav.content": "Contenuto",
"sidebar.nav.ai": "Assistente IA", "sidebar.nav.ai": "Assistente IA",
"sidebar.nav.technology": "Tecnologia",
"sidebar.nav.publishing": "Pubblicazione", "sidebar.nav.publishing": "Pubblicazione",
"sidebar.nav.data": "Dati", "sidebar.nav.data": "Dati",
"sidebar.nav.style": "Stile", "sidebar.nav.style": "Stile",
@@ -702,6 +712,9 @@
"settings.data.rebuildMediaLabel": "Ricostruisci database media", "settings.data.rebuildMediaLabel": "Ricostruisci database media",
"settings.data.rebuildMediaDescription": "Rianalizza tutti i file media e i metadati sidecar. Rigenera le voci mancanti.", "settings.data.rebuildMediaDescription": "Rianalizza tutti i file media e i metadati sidecar. Rigenera le voci mancanti.",
"settings.data.rebuildMediaAction": "Ricostruisci media", "settings.data.rebuildMediaAction": "Ricostruisci media",
"settings.data.rebuildScriptsLabel": "Ricostruisci database script",
"settings.data.rebuildScriptsDescription": "Rianalizza tutti gli script Python e ricostruisce lindice dei metadati degli script.",
"settings.data.rebuildScriptsAction": "Ricostruisci script",
"settings.data.rebuildLinksLabel": "Ricostruisci collegamenti post", "settings.data.rebuildLinksLabel": "Ricostruisci collegamenti post",
"settings.data.rebuildLinksDescription": "Rianalizza tutti i post e ricostruisce il grafo interno dei collegamenti tra post.", "settings.data.rebuildLinksDescription": "Rianalizza tutti i post e ricostruisce il grafo interno dei collegamenti tra post.",
"settings.data.rebuildLinksAction": "Ricostruisci collegamenti", "settings.data.rebuildLinksAction": "Ricostruisci collegamenti",

View File

@@ -54,6 +54,8 @@ export class PythonRuntimeManager {
private initializeDeferred: InitializeDeferred | null = null; private initializeDeferred: InitializeDeferred | null = null;
private ready = false; private ready = false;
private pendingRuns = new Map<string, PendingRun>(); private pendingRuns = new Map<string, PendingRun>();
private requestQueue: PythonWorkerRequest[] = [];
private activeRequestId: string | null = null;
private requestCounter = 0; private requestCounter = 0;
constructor(private readonly workerFactory: WorkerFactory = createPythonRuntimeWorker) {} constructor(private readonly workerFactory: WorkerFactory = createPythonRuntimeWorker) {}
@@ -123,7 +125,7 @@ export class PythonRuntimeManager {
entrypoint: options?.entrypoint, entrypoint: options?.entrypoint,
}; };
this.worker!.postMessage(message); this.enqueueRequest(message);
}); });
} }
@@ -162,7 +164,7 @@ export class PythonRuntimeManager {
cacheKey: options?.cacheKey, cacheKey: options?.cacheKey,
}; };
this.worker!.postMessage(message); this.enqueueRequest(message);
}); });
} }
@@ -198,7 +200,7 @@ export class PythonRuntimeManager {
cacheKey: options?.cacheKey, cacheKey: options?.cacheKey,
}; };
this.worker!.postMessage(message); this.enqueueRequest(message);
}); });
} }
@@ -234,7 +236,7 @@ export class PythonRuntimeManager {
cacheKey: options?.cacheKey, cacheKey: options?.cacheKey,
}; };
this.worker!.postMessage(message); this.enqueueRequest(message);
}); });
} }
@@ -262,6 +264,10 @@ export class PythonRuntimeManager {
const pendingRun = this.pendingRuns.get(payload.requestId); const pendingRun = this.pendingRuns.get(payload.requestId);
if (!pendingRun) { if (!pendingRun) {
if (this.activeRequestId === payload.requestId && payload.type !== 'stdout') {
this.activeRequestId = null;
this.dispatchNextRequest();
}
return; return;
} }
@@ -278,33 +284,40 @@ export class PythonRuntimeManager {
if (payload.type === 'runResult') { if (payload.type === 'runResult') {
if (pendingRun.kind !== 'run') { if (pendingRun.kind !== 'run') {
pendingRun.reject(new Error('Invalid response type for pending macro request')); pendingRun.reject(new Error('Invalid response type for pending macro request'));
this.finishRequest(payload.requestId);
return; return;
} }
pendingRun.resolve({ result: payload.result, stdout: pendingRun.stdout }); pendingRun.resolve({ result: payload.result, stdout: pendingRun.stdout });
this.finishRequest(payload.requestId);
return; return;
} }
if (payload.type === 'entrypoints') { if (payload.type === 'entrypoints') {
if (pendingRun.kind !== 'inspect-entrypoints') { if (pendingRun.kind !== 'inspect-entrypoints') {
pendingRun.reject(new Error('Invalid response type for pending run request')); pendingRun.reject(new Error('Invalid response type for pending run request'));
this.finishRequest(payload.requestId);
return; return;
} }
pendingRun.resolve(payload.entrypoints); pendingRun.resolve(payload.entrypoints);
this.finishRequest(payload.requestId);
return; return;
} }
if (payload.type === 'syntaxResult') { if (payload.type === 'syntaxResult') {
if (pendingRun.kind !== 'syntax-check') { if (pendingRun.kind !== 'syntax-check') {
pendingRun.reject(new Error('Invalid response type for pending syntax check request')); pendingRun.reject(new Error('Invalid response type for pending syntax check request'));
this.finishRequest(payload.requestId);
return; return;
} }
pendingRun.resolve({ errors: payload.errors }); pendingRun.resolve({ errors: payload.errors });
this.finishRequest(payload.requestId);
return; return;
} }
if (payload.type === 'macroResult') { if (payload.type === 'macroResult') {
if (pendingRun.kind !== 'macro-v1') { if (pendingRun.kind !== 'macro-v1') {
pendingRun.reject(new Error('Invalid response type for pending run request')); pendingRun.reject(new Error('Invalid response type for pending run request'));
this.finishRequest(payload.requestId);
return; return;
} }
@@ -314,10 +327,12 @@ export class PythonRuntimeManager {
} catch (error) { } catch (error) {
pendingRun.reject(error instanceof Error ? error : new Error(String(error))); pendingRun.reject(error instanceof Error ? error : new Error(String(error)));
} }
this.finishRequest(payload.requestId);
return; return;
} }
pendingRun.reject(new Error(payload.error)); pendingRun.reject(new Error(payload.error));
this.finishRequest(payload.requestId);
} }
private handleWorkerError(error: Error): void { private handleWorkerError(error: Error): void {
@@ -334,6 +349,8 @@ export class PythonRuntimeManager {
} }
this.pendingRuns.clear(); this.pendingRuns.clear();
this.requestQueue = [];
this.activeRequestId = null;
this.worker?.terminate(); this.worker?.terminate();
this.worker = null; this.worker = null;
this.initializingPromise = null; this.initializingPromise = null;
@@ -356,12 +373,50 @@ export class PythonRuntimeManager {
} }
this.pendingRuns.clear(); this.pendingRuns.clear();
this.requestQueue = [];
this.activeRequestId = null;
this.worker?.terminate(); this.worker?.terminate();
this.worker = null; this.worker = null;
this.initializingPromise = null; this.initializingPromise = null;
this.ready = false; this.ready = false;
} }
private enqueueRequest(request: PythonWorkerRequest): void {
if (!this.worker || !this.ready) {
this.requestQueue.push(request);
return;
}
if (this.activeRequestId !== null) {
this.requestQueue.push(request);
return;
}
this.activeRequestId = request.requestId;
this.worker.postMessage(request);
}
private dispatchNextRequest(): void {
if (!this.worker || !this.ready || this.activeRequestId !== null || this.requestQueue.length === 0) {
return;
}
const nextRequest = this.requestQueue.shift();
if (!nextRequest) {
return;
}
this.activeRequestId = nextRequest.requestId;
this.worker.postMessage(nextRequest);
}
private finishRequest(requestId: string): void {
if (this.activeRequestId === requestId) {
this.activeRequestId = null;
}
this.dispatchNextRequest();
}
private nextRequestId(): string { private nextRequestId(): string {
this.requestCounter += 1; this.requestCounter += 1;
return `req-${this.requestCounter}`; return `req-${this.requestCounter}`;

View File

@@ -63,7 +63,10 @@ describe('BlogmarkTransformService (Pyodide integration)', () => {
getScripts: async () => [createTransformScript()], getScripts: async () => [createTransformScript()],
}; };
const service = new BlogmarkTransformService({ provider }); const service = new BlogmarkTransformService({
provider,
resolvePythonRuntimeMode: async () => 'main-thread',
});
const result = await service.applyTransforms(createInput()); const result = await service.applyTransforms(createInput());

View File

@@ -247,67 +247,53 @@ describe('BlogmarkTransformService', () => {
expect(result.toasts).toEqual(['Step finished', 'Step finished']); expect(result.toasts).toEqual(['Step finished', 'Step finished']);
}); });
it('invokes python transform entrypoint with post payload shape', async () => { it('uses webworker executor when runtime mode resolves to webworker', async () => {
const globalsStore = new Map<string, unknown>(); const webworkerExecutor: BlogmarkTransformExecutor = {
const runPythonAsync = vi.fn(async (code: string) => { runTransform: vi.fn(async (_script, input) => ({ output: input.post, toasts: [] })),
if (code.includes('json.dumps(_result)')) { };
const payload = JSON.parse(String(globalsStore.get('__bds_transform_payload_json'))); const mainThreadExecutor: BlogmarkTransformExecutor = {
runTransform: vi.fn(async (_script, input) => ({ output: input.post, toasts: [] })),
if (code.includes('_transform_fn(_payload)')) {
return JSON.stringify(payload);
}
return JSON.stringify({
...payload.post,
title: 'Normalized',
categories: ['spielelog', 'asides'],
tags: ['inbox', 'spielen'],
});
}
return null;
});
vi.doMock('pyodide', () => ({
loadPyodide: vi.fn(async () => ({
globals: {
set: (key: string, value: unknown) => {
globalsStore.set(key, value);
},
},
runPythonAsync,
})),
}));
const provider: BlogmarkTransformScriptProvider = {
getScripts: vi.fn(async () => [
createScript({
id: 'pyodide-transform',
slug: 'pyodide-transform',
title: 'Pyodide Transform',
kind: 'transform',
entrypoint: 'normalize_blogmark',
content: 'def normalize_blogmark(post):\n return post',
}),
]),
}; };
const service = new BlogmarkTransformService({ provider }); const service = new BlogmarkTransformService({
provider: {
getScripts: async () => [createScript({ id: 'worker-script', slug: 'worker-script' })],
},
resolvePythonRuntimeMode: async () => 'webworker',
executors: {
webworker: webworkerExecutor,
'main-thread': mainThreadExecutor,
},
});
const result = await service.applyTransforms(createInput()); await service.applyTransforms(createInput());
const transformInvocationCode = runPythonAsync.mock.calls expect(webworkerExecutor.runTransform).toHaveBeenCalledTimes(1);
.map((call) => call[0]) expect(mainThreadExecutor.runTransform).not.toHaveBeenCalled();
.find((code) => typeof code === 'string' && String(code).includes('json.dumps(_result)')); });
expect(result.post.title).toBe('Normalized'); it('uses main-thread executor when runtime mode resolves to main-thread', async () => {
expect(result.post.categories).toEqual(['spielelog', 'asides']); const webworkerExecutor: BlogmarkTransformExecutor = {
expect(result.post.tags).toEqual(['inbox', 'spielen']); runTransform: vi.fn(async (_script, input) => ({ output: input.post, toasts: [] })),
expect(transformInvocationCode).toBeDefined(); };
expect(String(transformInvocationCode)).not.toContain('import inspect'); const mainThreadExecutor: BlogmarkTransformExecutor = {
expect(String(transformInvocationCode)).toContain('\ntry:\n'); runTransform: vi.fn(async (_script, input) => ({ output: input.post, toasts: [] })),
expect(String(transformInvocationCode)).toContain('\nexcept TypeError:\n'); };
expect(String(transformInvocationCode)).not.toContain('\n try:\n');
expect(String(transformInvocationCode)).not.toContain('\n except TypeError:\n'); const service = new BlogmarkTransformService({
provider: {
getScripts: async () => [createScript({ id: 'main-script', slug: 'main-script' })],
},
resolvePythonRuntimeMode: async () => 'main-thread',
executors: {
webworker: webworkerExecutor,
'main-thread': mainThreadExecutor,
},
});
await service.applyTransforms(createInput());
expect(mainThreadExecutor.runTransform).toHaveBeenCalledTimes(1);
expect(webworkerExecutor.runTransform).not.toHaveBeenCalled();
}); });
}); });

View File

@@ -237,6 +237,42 @@ describe('GitEngine', () => {
}); });
}); });
describe('getChangedScriptFilesBetween', () => {
it('returns added, modified, deleted and renamed script file changes from name-status output', async () => {
mockRaw.mockResolvedValue([
'M', 'scripts/existing.py',
'A', 'scripts/new_script.py',
'D', 'scripts/removed.py',
'R100', 'scripts/old_name.py', 'scripts/new_name.py',
'M', 'posts/2026/02/ignored.md',
].join('\0'));
const result = await gitEngine.getChangedScriptFilesBetween('/tmp/project', 'before', 'after');
expect(mockRaw).toHaveBeenCalledWith([
'diff',
'--name-status',
'--find-renames',
'-z',
'before..after',
'--',
'scripts',
]);
expect(result).toEqual([
{ status: 'modified', path: 'scripts/existing.py' },
{ status: 'added', path: 'scripts/new_script.py' },
{ status: 'deleted', path: 'scripts/removed.py' },
{ status: 'renamed', path: 'scripts/new_name.py', previousPath: 'scripts/old_name.py' },
]);
});
it('returns empty changes when refs are empty or identical', async () => {
expect(await gitEngine.getChangedScriptFilesBetween('/tmp/project', 'same', 'same')).toEqual([]);
expect(await gitEngine.getChangedScriptFilesBetween('/tmp/project', ' ', 'after')).toEqual([]);
});
});
describe('getCommitDiffContent', () => { describe('getCommitDiffContent', () => {
it('should return commit patch text in diff content shape', async () => { it('should return commit patch text in diff content shape', async () => {
mockShow.mockResolvedValue([ mockShow.mockResolvedValue([

View File

@@ -739,6 +739,30 @@ describe('MetaEngine', () => {
expect((metadata as any)?.blogmarkCategory).toBe('article'); expect((metadata as any)?.blogmarkCategory).toBe('article');
}); });
it('should set and get pythonRuntimeMode in project metadata', async () => {
await metaEngine.setProjectMetadata({
name: 'My Blog',
pythonRuntimeMode: 'main-thread',
} as any);
const metadata = await metaEngine.getProjectMetadata();
expect((metadata as any)?.pythonRuntimeMode).toBe('main-thread');
});
it('should persist pythonRuntimeMode to filesystem', async () => {
await metaEngine.setProjectMetadata({
name: 'Runtime Mode Project',
pythonRuntimeMode: 'webworker',
} as any);
const metaDir = metaEngine.getMetaDir();
const projectPath = normalizePath(`${metaDir}/project.json`);
const content = mockFiles.get(projectPath);
const parsed = JSON.parse(content!);
expect(parsed.pythonRuntimeMode).toBe('webworker');
});
it('should persist blogmarkCategory to filesystem', async () => { it('should persist blogmarkCategory to filesystem', async () => {
await metaEngine.setProjectMetadata({ await metaEngine.setProjectMetadata({
name: 'Test Project', name: 'Test Project',

View File

@@ -55,6 +55,23 @@ vi.mock('uuid', () => ({
})); }));
vi.mock('fs/promises', () => ({ vi.mock('fs/promises', () => ({
readdir: vi.fn(async (dirPath: string, options?: { withFileTypes?: boolean }) => {
if (options?.withFileTypes) {
const files = Array.from((globalThis as any).__mockScriptFiles.keys()) as string[];
const names = files
.filter((filePath) => filePath.startsWith(`${dirPath}/`))
.map((filePath) => filePath.slice(dirPath.length + 1))
.filter((name) => !name.includes('/'));
return names.map((name) => ({
name,
isDirectory: () => false,
isFile: () => true,
}));
}
return [];
}),
readFile: vi.fn(async (filePath: string) => { readFile: vi.fn(async (filePath: string) => {
const value = (globalThis as any).__mockScriptFiles.get(filePath); const value = (globalThis as any).__mockScriptFiles.get(filePath);
if (typeof value !== 'string') { if (typeof value !== 'string') {
@@ -175,4 +192,98 @@ describe('ScriptEngine', () => {
expect(loaded?.title).toBe('Metadata Test'); expect(loaded?.title).toBe('Metadata Test');
expect(loaded?.entrypoint).toBe('render'); expect(loaded?.entrypoint).toBe('render');
}); });
it('rebuilds scripts from filesystem and applies external file metadata/content', async () => {
const scriptPath = '/mock/userData/projects/default/scripts/external_transform.py';
mockFiles.set(scriptPath, [
'"""',
'---',
'id: "external-script-id"',
'projectId: "default"',
'slug: "external_transform"',
'title: "External Transform"',
'kind: "transform"',
'entrypoint: "transform"',
'enabled: false',
'version: 3',
'createdAt: "2026-02-20T10:00:00.000Z"',
'updatedAt: "2026-02-21T11:00:00.000Z"',
'---',
'"""',
'def transform(context):',
' return context',
].join('\n'));
await scriptEngine.rebuildDatabaseFromFiles();
const all = await scriptEngine.getAllScripts();
expect(all).toHaveLength(1);
expect(all[0].id).toBe('external-script-id');
expect(all[0].slug).toBe('external_transform');
expect(all[0].kind).toBe('transform');
expect(all[0].entrypoint).toBe('transform');
expect(all[0].enabled).toBe(false);
expect(all[0].version).toBe(3);
expect(all[0].title).toBe('External Transform');
expect(all[0].content).toContain('def transform(context):');
});
it('reconciles git changes for scripts (modify/add/delete)', async () => {
const created = await scriptEngine.createScript({
title: 'Render Hero',
kind: 'macro',
content: 'def render(context):\n return {"html": "<h1>Hi</h1>"}',
});
const existingPath = '/repo/scripts/render_hero.py';
mockFiles.set(existingPath, [
'"""',
'---',
`id: "${created.id}"`,
'projectId: "default"',
'slug: "render_hero"',
'title: "Render Hero Updated Outside"',
'kind: "macro"',
'entrypoint: "render"',
'enabled: true',
'version: 8',
'createdAt: "2026-02-20T10:00:00.000Z"',
'updatedAt: "2026-02-21T11:00:00.000Z"',
'---',
'"""',
'def render(context):',
' return {"html": "<h1>Outside</h1>"}',
].join('\n'));
const addedPath = '/repo/scripts/new_transform.py';
mockFiles.set(addedPath, [
'"""',
'---',
'id: "added-script-id"',
'projectId: "default"',
'slug: "new_transform"',
'title: "New Transform"',
'kind: "transform"',
'entrypoint: "transform"',
'enabled: true',
'version: 1',
'createdAt: "2026-02-22T10:00:00.000Z"',
'updatedAt: "2026-02-22T11:00:00.000Z"',
'---',
'"""',
'def transform(context):',
' return context',
].join('\n'));
const result = await scriptEngine.reconcileScriptsFromGitChanges('/repo', [
{ status: 'modified', path: 'scripts/render_hero.py' },
{ status: 'added', path: 'scripts/new_transform.py' },
{ status: 'deleted', path: 'scripts/render_hero.py' },
]);
expect(result.updated).toBe(1);
expect(result.created).toBe(1);
expect(result.deleted).toBe(1);
expect(result.processedFiles).toBe(3);
});
}); });

View File

@@ -166,12 +166,15 @@ const mockScriptEngine = {
deleteScript: vi.fn(), deleteScript: vi.fn(),
getScript: vi.fn(), getScript: vi.fn(),
getAllScripts: vi.fn(), getAllScripts: vi.fn(),
rebuildDatabaseFromFiles: vi.fn(),
reconcileScriptsFromGitChanges: vi.fn(),
}; };
const mockGitEngine = { const mockGitEngine = {
checkAvailability: vi.fn(), checkAvailability: vi.fn(),
getHeadCommit: vi.fn(), getHeadCommit: vi.fn(),
getChangedPostFilesBetween: vi.fn(), getChangedPostFilesBetween: vi.fn(),
getChangedScriptFilesBetween: vi.fn(),
getRepoState: vi.fn(), getRepoState: vi.fn(),
getStatus: vi.fn(), getStatus: vi.fn(),
getDiff: vi.fn(), getDiff: vi.fn(),
@@ -575,12 +578,21 @@ describe('IPC Handlers', () => {
{ status: 'modified', path: 'posts/2026/02/existing.md' }, { status: 'modified', path: 'posts/2026/02/existing.md' },
{ status: 'added', path: 'posts/2026/02/new-post.md' }, { status: 'added', path: 'posts/2026/02/new-post.md' },
]); ]);
mockGitEngine.getChangedScriptFilesBetween.mockResolvedValue([
{ status: 'modified', path: 'scripts/transform.py' },
]);
mockPostEngine.reconcilePublishedPostsFromGitChanges.mockResolvedValue({ mockPostEngine.reconcilePublishedPostsFromGitChanges.mockResolvedValue({
created: 1, created: 1,
updated: 1, updated: 1,
deleted: 0, deleted: 0,
processedFiles: 2, processedFiles: 2,
}); });
mockScriptEngine.reconcileScriptsFromGitChanges.mockResolvedValue({
created: 0,
updated: 1,
deleted: 0,
processedFiles: 1,
});
const result = await invokeHandler('git:pull', '/repo'); const result = await invokeHandler('git:pull', '/repo');
@@ -588,10 +600,14 @@ describe('IPC Handlers', () => {
expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo'); expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo');
expect(mockGitEngine.getHeadCommit).toHaveBeenNthCalledWith(2, '/repo'); expect(mockGitEngine.getHeadCommit).toHaveBeenNthCalledWith(2, '/repo');
expect(mockGitEngine.getChangedPostFilesBetween).toHaveBeenCalledWith('/repo', 'before-head', 'after-head'); expect(mockGitEngine.getChangedPostFilesBetween).toHaveBeenCalledWith('/repo', 'before-head', 'after-head');
expect(mockGitEngine.getChangedScriptFilesBetween).toHaveBeenCalledWith('/repo', 'before-head', 'after-head');
expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).toHaveBeenCalledWith('/repo', [ expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).toHaveBeenCalledWith('/repo', [
{ status: 'modified', path: 'posts/2026/02/existing.md' }, { status: 'modified', path: 'posts/2026/02/existing.md' },
{ status: 'added', path: 'posts/2026/02/new-post.md' }, { status: 'added', path: 'posts/2026/02/new-post.md' },
]); ]);
expect(mockScriptEngine.reconcileScriptsFromGitChanges).toHaveBeenCalledWith('/repo', [
{ status: 'modified', path: 'scripts/transform.py' },
]);
expect(result).toEqual({ success: true }); expect(result).toEqual({ success: true });
}); });
@@ -603,7 +619,9 @@ describe('IPC Handlers', () => {
expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo'); expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo');
expect(mockGitEngine.getChangedPostFilesBetween).not.toHaveBeenCalled(); expect(mockGitEngine.getChangedPostFilesBetween).not.toHaveBeenCalled();
expect(mockGitEngine.getChangedScriptFilesBetween).not.toHaveBeenCalled();
expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).not.toHaveBeenCalled(); expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).not.toHaveBeenCalled();
expect(mockScriptEngine.reconcileScriptsFromGitChanges).not.toHaveBeenCalled();
expect(result).toEqual({ success: false, code: 'conflict' }); expect(result).toEqual({ success: false, code: 'conflict' });
}); });
@@ -617,7 +635,9 @@ describe('IPC Handlers', () => {
expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo'); expect(mockGitEngine.pull).toHaveBeenCalledWith('/repo');
expect(mockGitEngine.getChangedPostFilesBetween).not.toHaveBeenCalled(); expect(mockGitEngine.getChangedPostFilesBetween).not.toHaveBeenCalled();
expect(mockGitEngine.getChangedScriptFilesBetween).not.toHaveBeenCalled();
expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).not.toHaveBeenCalled(); expect(mockPostEngine.reconcilePublishedPostsFromGitChanges).not.toHaveBeenCalled();
expect(mockScriptEngine.reconcileScriptsFromGitChanges).not.toHaveBeenCalled();
expect(result).toEqual({ success: true }); expect(result).toEqual({ success: true });
}); });
}); });
@@ -2712,6 +2732,23 @@ describe('IPC Handlers', () => {
expect(result).toEqual(expected); expect(result).toEqual(expected);
}); });
}); });
describe('scripts:rebuildFromFiles', () => {
it('should set project context and trigger ScriptEngine rebuild', async () => {
mockProjectEngine.getActiveProject.mockResolvedValue({
id: 'project-1',
dataPath: '/external/data',
});
mockProjectEngine.getDataDir.mockReturnValue('/resolved/project-data');
mockScriptEngine.rebuildDatabaseFromFiles.mockResolvedValue(undefined);
const result = await invokeHandler('scripts:rebuildFromFiles');
expect(mockScriptEngine.setProjectContext).toHaveBeenCalledWith('project-1', '/resolved/project-data');
expect(mockScriptEngine.rebuildDatabaseFromFiles).toHaveBeenCalled();
expect(result).toBe(true);
});
});
}); });
// ============ Error Handling ============ // ============ Error Handling ============

View File

@@ -117,6 +117,33 @@ describe('SettingsView Diff Preferences', () => {
); );
}); });
it('includes python runtime mode in metadata save payload', async () => {
(window as any).electronAPI.meta.getProjectMetadata = vi.fn().mockResolvedValue({
maxPostsPerPage: 75,
publicUrl: 'https://example.com',
pythonRuntimeMode: 'main-thread',
categorySettings: {
article: { renderInLists: true, showTitle: true },
picture: { renderInLists: true, showTitle: true },
aside: { renderInLists: true, showTitle: false },
page: { renderInLists: false, showTitle: true },
},
});
render(<SettingsView />);
await screen.findByDisplayValue('Main Thread (Legacy)');
const saveButton = screen.getByRole('button', { name: /save project settings/i });
fireEvent.click(saveButton);
await new Promise((resolve) => setTimeout(resolve, 0));
expect((window as any).electronAPI.meta.updateProjectMetadata).toHaveBeenCalledWith(
expect.objectContaining({ pythonRuntimeMode: 'main-thread' })
);
});
it('renders category settings checkboxes with required defaults', async () => { it('renders category settings checkboxes with required defaults', async () => {
render(<SettingsView />); render(<SettingsView />);
@@ -131,6 +158,26 @@ describe('SettingsView Diff Preferences', () => {
expect((articleShowTitle as HTMLInputElement).checked).toBe(true); expect((articleShowTitle as HTMLInputElement).checked).toBe(true);
}); });
it('triggers scripts rebuild from data maintenance section', async () => {
const rebuildScriptsMock = vi.fn().mockResolvedValue(undefined);
(window as any).electronAPI = {
...(window as any).electronAPI,
scripts: {
...(window as any).electronAPI?.scripts,
rebuildFromFiles: rebuildScriptsMock,
},
};
render(<SettingsView />);
const rebuildScriptsButton = await screen.findByRole('button', { name: /rebuild scripts/i });
fireEvent.click(rebuildScriptsButton);
await new Promise((resolve) => setTimeout(resolve, 0));
expect(rebuildScriptsMock).toHaveBeenCalledTimes(1);
});
it('persists category settings changes via project metadata update', async () => { it('persists category settings changes via project metadata update', async () => {
render(<SettingsView />); render(<SettingsView />);

View File

@@ -191,6 +191,32 @@ describe('PythonRuntimeManager', () => {
await expect(runPromise).rejects.toThrow('boom'); await expect(runPromise).rejects.toThrow('boom');
}); });
it('queues concurrent execute calls and sends the next request only after completion', async () => {
const worker = new MockWorker();
const manager = new PythonRuntimeManager(() => worker as unknown as Worker);
const initPromise = manager.initialize();
worker.emitMessage({ type: 'ready' });
await initPromise;
const firstRun = manager.execute('1 + 1');
const secondRun = manager.execute('2 + 2');
await Promise.resolve();
expect(worker.postedMessages).toHaveLength(1);
const firstRequest = worker.postedMessages[0] as { requestId: string };
worker.emitMessage({ type: 'runResult', requestId: firstRequest.requestId, result: '2' });
await expect(firstRun).resolves.toEqual({ result: '2', stdout: '' });
await Promise.resolve();
expect(worker.postedMessages).toHaveLength(2);
const secondRequest = worker.postedMessages[1] as { requestId: string };
worker.emitMessage({ type: 'runResult', requestId: secondRequest.requestId, result: '4' });
await expect(secondRun).resolves.toEqual({ result: '4', stdout: '' });
});
it('terminates timed out worker and recovers with a new worker', async () => { it('terminates timed out worker and recovers with a new worker', async () => {
const workers: MockWorker[] = []; const workers: MockWorker[] = [];
const manager = new PythonRuntimeManager(() => { const manager = new PythonRuntimeManager(() => {