CHANGELOG.md

Clips Kitty SDK changelog

No version is on PyPI yet. Clips Kitty bundles the SDK from the commit it is built from, so a plugin always runs with the SDK of the Clips Kitty it runs in.

The installed app puts its own SDK on the plugin's path, and it wins over one installed with pip. A plugin that uses a module added in a later SDK sets requires.clips_kitty to the first release that bundles it (Versioning).

Unreleased

• media.jpeg says "there is no frame at … s in the video: is it that long?" for a time past the end with FFmpeg 7 and later too. Those end the read with an error ("received no packets") where older FFmpeg wrote nothing, so the plugin was told FFmpeg couldn't read the video.

1.3.0 (in Clips Kitty 2.1.0)

• The edit step. A plugin with edits in its outputs and moments in its inputs suggests edits for the clips Clips Kitty is about to make, in a run of its own (steps: ["edit"]). Clips Kitty keeps each suggestion for the creator, who can use it in the editor; it changes no clip by itself. contract.STEPS gains edit, contract.PLANNED_STEPS is ("export",), manifest.OUTPUTS gains edits, and manifest.PLANNED_OUTPUTS is ("clips",). job.wants("edit") is True in an edit run.
• The validator: edits in outputs without moments in inputs is refused with "edits are suggested for the clips Clips Kitty makes from moments: add moments to inputs", and a plugin given moments may answer with edits ("add ratings, context or edits to outputs"). A manifest still gets at most one of these errors. step_problem says "can't suggest edits for clips: its manifest needs moments in inputs and edits in outputs", and points a plugin that only edits to Suggest edits rather than Rate & understand.
• job.suggest_edit(m) returns the clip's EditSuggestion, the same one on every call, whose methods chain: cut, trim, keep_only, mute, volume, fade, speed, title_overlay (the editor's Hook title), crop and reason, with cuts, mutes and length to read back. Times are seconds of the video. A value outside the render's limits raises ContractError at the call; a span outside the clip, a layout the job's clips don't use, and a fade, speed or hook title length the editor doesn't offer are logged once. In a run not asked to edit, or on the plugin's own range, the call is logged and nothing is kept.
• job.limits.crops lists the layouts the job's clips can use (contract.CROPS: track, center, letterbox, or none). Moment.suggested holds what edit plugins before this one suggested for the clip (Suggested); without transcript.read their reasons and hook titles are left out.
• check_result checks an answer's edit in an edit run: at most 20 cuts and 20 mutes, volume 0 to 2, fades 0 to 3 s, speed 0.5 to 3, a hook title of 1 to 120 characters for 1 to 10 s, a layout of at most 32 characters and a reason of at most 160. A problem refuses the whole answer; a field it doesn't name is ignored.
• host.read_edits() reads an edit run's answer as Clips Kitty keeps it: cuts and mutes clamped to the clip and joined, cuts that would leave less than the job's shortest clip ignored, fades, speed and the hook title's seconds set to the nearest of the editor's choices (contract.FADE_CHOICES, SPEED_CHOICES, HOOK_SECONDS_CHOICES) with a changed: line, the hook title on one line without web addresses, and a layout the job's clips don't use ignored. Every field it doesn't take gets an ignored: line. read_answers says "ignored: edits, because this run wasn't asked to suggest edits".
• host.clean_note(), and the reason and hook title read_edits keeps, drop invisible format characters (a zero-width space, a word joiner, a right-to-left override) before taking out web addresses, so none can hide one. A zero-width joiner or non-joiner between two non-ASCII characters stays, for emoji and the scripts that need it.
• run --steps edit runs an edit run, alone; with another step it is refused. A plugin that only suggests edits gets an edit run without --steps, and one that also finds, rates or understands is told it can be run again with --steps edit. --layout (standard, the default, or podcast, sports, gaming, vertical-live, whole-frame) fills limits.crops as the app would. --moments items may carry suggested. Each clip's line shows its length before and after and what is suggested.
• testing.run_plugin(..., steps=["edit"], layout="standard") returns PluginRun.edits, each clip's suggestion as Clips Kitty keeps it; make_job takes layout too.
• new --template editor, a sixth template: it suggests edits for the clips Clips Kitty makes, by the words said in them. It mutes the words the creator lists in mute_words, cuts the wait after "respawn timer" and adds the hook title "Quark burst!" where "quark burst" is said, for Quarkbloom Arena (a made-up game). Like the other templates it has its own tests and workflow, and it needs no FFmpeg.
• Steps explains the edit step: what a plugin may suggest, how Clips Kitty fits it and what each changed: and ignored: line means, with an editor that the SDK's documentation tests run as written. A plugin that suggests edits needs 1.3.0: don't ship an older copy of the SDK inside a plugin, which comes first on the path and has no job.suggest_edit.

1.2.0 (in Clips Kitty 2.1.0)

• python -m clipskitty_sdk --version prints the SDK's version and the plugin contract it follows: clipskitty-sdk 1.2.0 (plugin contract 1).
• Installed with pip, the SDK adds a clipskitty-sdk command, the same as python -m clipskitty_sdk.
• A test extra installs pytest, for a plugin's own tests: pip install "clipskitty-sdk[yaml,test]".
• The package ships py.typed, so type checkers read its type hints.
• A wheel built from a folder where tests have run leaves out Python's byte-code caches (*.pyc).
• The package's details list the Python versions it supports, and link to its issues and this changelog.
• edits is a planned output, for the edit step: outputs: [edits] is refused with "output 'edits' is planned, not supported by plugin API 1" instead of "unknown output". contract.PLANNED_STEPS names the planned steps, edit and export.
• validate and run show the line of clipskitty.yaml each problem is on, and "did you mean permissions?" under a misspelt field (Report.hints, manifest.line_marks). Without a clipskitty.yaml, or without PyYAML on the PC, they say so and exit with 2.
• version: 0.1 is refused with "YAML read this as the number 0.1; write a version like 0.1.0", in the app too. The run.python_requirements warning says that pipelines run on Clips Kitty's own Python.
• validate and run warn about code that would fail on Clips Kitty's own Python (clipskitty_sdk.lint): syntax newer than Python 3.11, imports it doesn't promise (other packages, Clips Kitty's own code, modules the app leaves out or Windows lacks, a module at the plugin's root), and text files opened without encoding=. Files under tests/ are skipped.
• host.APP_PYTHON is (3, 11), the Python Clips Kitty runs plugins on. host.plugin_env sets PYTHONUTF8=0.
• run stops a plugin when Clips Kitty would (host.timeout_seconds: run.timeout_minutes, else 60 minutes to find and 10 to understand or rate) instead of after 600 seconds; --timeout still sets another limit.
• run: a refused run makes no job folder; --set values follow the setting's type, and an unknown one lists the plugin's settings; --steps takes several words (--steps understand rate), and --steps edit says edit is planned; --game, --game-hint and --model fill video.games and job.models; a warning when the plugin asks for FFmpeg and none is found; each moment's reason (why:), and a warning for a label that isn't in the manifest's events; on a terminal, progress rewrites one line.
• run(main) started without a job folder (python src/main.py) says how to try the plugin and exits with 2. job.settings[name] for a setting with no value raises SettingMissing: creators see a plain line and the log gets the developer's hint. For a secret the creator set, which is never in job.settings, the hint says to read it with job.secret(name), and creators see the line for a mistake in the plugin's code. A slip in the plugin's own code (a KeyError, TypeError and the like) gives creators "it stopped on a mistake in its own code. Ask its developer to fix it.", with the exception and its line in the log; other exceptions keep their message.
• job.rate and job.add_range show the score they refused: "(got 140)".
• clipskitty_sdk.devrun holds the core of run, so other tools can run a plugin the same way.
• python -m clipskitty_sdk sample OUT.mp4 writes a 40-second test video, made only with FFmpeg's own generators (a red banner at the top and a loud sound from 22 to 27 s, scene cuts at 10, 20 and 30 s), and OUT.transcript.json, which says "quark burst" from 21 to 26 s. It refuses to write inside a plugin's folder, which Clips Kitty copies on install, and its example of where to write instead is outside the plugin's folder, even when run from src/. A path it can't write to is an error line, not a traceback, and leaves no half-written file.
• run --sample makes that video and transcript in the job folder and runs the plugin on them. A plugin without video.read or ffmpeg needs no FFmpeg for it: it gets the transcript and a 40-second length. A find run of a plugin without video.read no longer needs --video: its moments are fitted to --duration, else to the end of --transcript.
• python -m clipskitty_sdk frame VIDEO --at SECONDS writes one frame as a PNG; with --region "0.30,0.10,0.40,0.10" it draws a box around that part of the screen and prints it in pixels. --region takes the numbers as one argument or several. It reads 10-bit (HDR) videos too. Like sample, it refuses to write inside a plugin's folder.
• clipskitty_sdk.samples: sample_transcript(), SAMPLE_VIDEO_SECONDS, make_sample(), parse_region(), region_pixels() and write_frame().
• clipskitty_sdk.testing, for a plugin's own tests: run_plugin() runs the plugin as run does and returns what it did (ok, error, moments, answers, notes, events, log); make_job() writes the job folder without running anything; sample_video() makes the sample video, and skips a pytest test when there is no FFmpeg; sample_transcript() and SAMPLE_VIDEO_SECONDS need nothing. A run that can't start raises ContractError. Both take ollama_host, ollama_model and models, as run's --ollama-host, --ollama-model and --model do, and run_plugin() takes secrets, as --secret does.
• devrun.moments_from() and devrun.transcript_from() take moments and a transcript as data rather than files.
• clipskitty_sdk.media reads the video with the FFmpeg and FFprobe Clips Kitty hands over: Region (a part of the screen as fractions of the frame; Region.parse("0.30,0.10,0.40,0.10"), .pixels()), probe(), frames() (one region's frames, shrunk, at the rate asked), jpeg(), loudness() (per second) and scene_cuts(). Without ffmpeg or video.read in the permissions it says which to add (MediaError). loudness() and scene_cuts() are adapted from the MIT scene-cut example pipeline.
• clipskitty_sdk.signals turns numbers and frames into moments: colour_share(), brightness(), difference(), spikes() (against a rolling median), stretches(), merge(), and around(), which puts lead-in and tail around a moment and fits it to the job's limits and the video's length.
• clipskitty_sdk.text: words_of() reads a comma-separated words setting, find_words() matches whole words in any case or spacing, said() says when each word is said (by the words' own times when the transcript has them) and hits() which words are said during a moment.
• clipskitty_sdk.local_model asks the creator's local model, on this PC only: model(), ask() (with pictures, and JSON with thinking off) and can_see(). It refuses without the ollama permission, when no local model is set (Clips Kitty's AI runs at a cloud provider), and at an address that isn't 127.0.0.1, localhost or ::1; it never falls back to another address. run says when a plugin with the ollama permission is given no --ollama-model.
• Requests to this PC ignore proxy settings: local_model's, and LocalAPI's when its address is 127.0.0.1, localhost or ::1. Before, a proxy set in the environment received them. They follow a redirect only to the same address (scheme, host and port). LocalAPI at any other address uses proxy settings as before.
• python -m clipskitty_sdk new FOLDER --template NAME starts a plugin from one of five templates: blank, transcript, game-events, rater and understander (new --list says what each does). --publisher, --name, --game, --author and --license fill in the plugin's details; on a terminal it asks for a missing publisher and name, and elsewhere it uses your-github-name and the folder's name. It writes only into a new or empty folder, and checks the clipskitty.yaml it makes before writing anything. Each plugin it makes has src/main.py, README.md, tests/test_main.py (using clipskitty_sdk.testing), CHANGELOG.md, .gitignore, LICENSE (MIT; another --license leaves it out and says to add one), TEMPLATE-LICENSE.txt (the SDK's MIT notice, for the code that came from the template) and a GitHub workflow that validates the plugin and runs its tests on Clips Kitty's Python (3.11), on Linux and Windows. The templates ship inside the package (clipskitty_sdk.scaffold). After making a plugin, new links Your first game pipeline.
• run(main): importing a part of the SDK that the SDK running the plugin doesn't have (such as from clipskitty_sdk import media on an older one) gives "This pipeline needs a newer version of Clips Kitty. Update Clips Kitty, or ask the pipeline's developer which version it needs." (job.NEWER_VERSION, job.needs_newer_sdk()), with the import error in the log. Clips Kitty says the same when such an import is at the top of the plugin's file, instead of "needs clipskitty_sdk".
• python -m clipskitty_sdk install FOLDER puts a plugin into the Clips Kitty running on this PC, as Marketplace › Browse › For developers does: it prints Clips Kitty's own install screen as text and asks first (clipskitty_sdk.installer). --yes skips the question only when nothing is new (the same plugin is installed and the update adds no permission, network host, data sent, change to where it runs, or step), and --watch reinstalls it on each save until a save adds something. It refuses .venv, venv and node_modules inside the folder and symbolic links, says when Clips Kitty isn't running or came out before plugin support, reads the session secret from --data-dir, the file Clips Kitty names or the installed app's data folder, and talks only to Clips Kitty on this PC, ignoring proxy settings. A mistyped --api, an answer that isn't HTTP and one cut off part way are error lines, not tracebacks. LocalAPI never sends the session secret.
• python -m clipskitty_sdk listing FOLDER --section SECTION writes the file that lists a finished plugin in Awesome Clips Kitty, the catalog the Marketplace reads (clipskitty_sdk.listing): its id, repository, path, section, --alias words, the date and this version at its full commit, with its tag when v<version> points there. --to adds the version to a listing that exists and refuses one already listed. It refuses a folder with changes that aren't committed, a commit not on a pushed branch, a repository other than the manifest's, an id whose publisher doesn't own the repository, and a template's placeholders. It runs only git commands that read and fetches nothing, then says to open the pull request with the catalog's index rebuilt, which CI checks. When git can't read the repository (one owned by another user, say), it shows git's own words and the command git suggests.
• manifest.yaml_data() reads a YAML text with PyYAML's safe loader, for the SDK's other YAML files.

1.1.0

• Plugins can understand and rate moments that were found before them, as well as find their own (Steps).
• job.moments: the moments handed over to understand or rate, as Moments.
• job.understand(moment, text) and job.rate(moment, score, reason): a run's answers.
• job.wants(step): whether this run is asked to find, understand or rate.
• job.text(moment): what is said during a moment.
• Moment, exported from clipskitty_sdk.
• run --steps and --moments try a moment run; --duration and --min-score go with them, and --video is needed only to find moments or for a plugin with video.read.

1.0.0

• The job contract: job.json in, progress lines out, result.json back (clipskitty_sdk.contract, check_job, check_result).
• read_job(), run(main) and Job: progress, log, secret, add_range, finish and fail.
• The manifest checks the app, the plugin manager and the registry run (clipskitty_sdk.manifest).
• clipskitty_sdk.host: the job folder, the process and the result, as the app runs a plugin.
• The validate, schema and run commands.
• LocalAPI: a small client for Clips Kitty's local API.