ResearchUpdated
Building Rothko, Plnty's creative agent
Requirement 1
To learn the intuitive
Understanding how to communicate the intuitive elements and subjective preferences that characterise Yambo Studio's films, such as timing and camera movement choices, is a challenge.
Over the course of dozens of product films, Yambo Studio has developed certain unspoken formulas for camera and animation work. The movement speeds up as it arrives at a cut, carrying through to the next shot; it then slows to linger on the product.
You can see examples of this formula with the foil package below. When attempting to translate the same motion principles to the kids camera film, words alone failed to achieve a good result. However, using the foil film as a motion reference, the new film began to behave more in line with my preferences.
At Plnty, we performed an optical flow analysis of the video to measure movement between frames and compare the films. This data was then used to establish a default behaviour for Rothko, an applicable guideline for camera motion that dictates the type of camera motion expected in this given context. Then you can make it faster or slower using language.
Requirement 2
Then, a creativity ladder
If we at Plnty are to equip an agent with the capacity to not only create, but to direct creativity, then we need a way to approximate a designer's ability to judge the appropriateness of a solution compared against a brief and its context. This judgement is informed largely by ineffable qualities such as 'mood'. For example, certain beauty brands might favour an avant-garde product composition where others require something more conventional. If Rothko cannot determine the suitable response to a brief (on a spectrum from 'playing it safe' through to 'tear up the book'), then output will be uninformed and arbitrary. The solution: a 'Creativity Ladder'. The further up the ladder, the more extreme the ideas. The ladder offers six levels of freedom over composition, scale and staging.
These early tests are built on the same prompt at each level. At the low end of the ladder, Rothko writes a plain brief. From the middle up, it draws on the Plnty team's taste, with bolder staging at the top. Your brief still sets the boundaries and allows an override for projects requiring a departure from studio convention.
| Six steps | Very low | Low | Medium | Medium high | High | Very high |
|---|---|---|---|---|---|---|
| From wordsPerfume launch"a launch picture for a new perfume called North" | ![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| EditDance festival poster"turn this into a poster for a dance festival", on Sasha's dancer | ![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| EditPerfume campaign"make a campaign picture for this perfume", on Sasha's bottle | ![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| From wordsRunning shoes launch"a launch picture for new running shoes" | ![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
Requirement 3
Creative skills
A creative skill in Plnty is a working process, or series of tools combined into a workflow, that Rothko can recall on command and apply to any chosen context. The skill is a pre-configured set of steps which remain in a fixed order. Choosing and developing output in a new context using a skill allows for interesting results with an element of control depending on the intended remit of the skill.
Select something (or many things) on your board to open Rothko, and select Run a creative skill. That starts the whole job, with three seconds to cancel using Esc before it spends credits. You can stop a run at any time. Turn on Spending limits in Settings, under Creative Agent, to require approval above your limit.
Transcript
No words. The sounds of the kitchen for the first twelve seconds, the flour and the eggs, then quiet to the end.
Requirement 4
Creating Rothko's 'brain'
Rothko uses separate models to plan, write instructions and judge results. At Plnty, our corrections and examples guide the planning and writing. The judge compares pictures against your brief and the criteria for that stage. Bringing our taste more directly into those judgments is work we still have to do.
In a session recorded for this paper, Rothko kept three of eight first results, one from each of three directions. It combined these selects into another picture, then began developing all four. We still need to measure how often those choices lead to work a team actually uses.
Until the processes can be connected so that decisions are informed by other phases of the decision-making process, the workflow is going to be scattered.
From Yam
A lifetime of creative work teaches you what to explore, what to keep, and when to change direction. Over the past few months, I've been pouring that experience into Rothko, giving it a sense of how creative decisions get made, and what makes an idea worth pursuing.
Automating the Creative Workflow
At Plnty, we've written about this in 'Automating the Creative Workflow'. The paper describes what we've built, the decisions we've recorded and what we still need to test. It also looks beyond today's skills, toward an agent that can choose which stage the work needs next.
The draft is confidential for now. Sign in and request access; we'll email you when it's available to your account.
More hands for a small team
I want a small creative team to have more room to explore an idea before committing to it. Rothko can take on the combinations, variations and first clips, leaving the team more time to choose what deserves attention as it moves towards a finished product.
Sources and methodology
Rothko's practice draws on the works below, as cited in Plnty's research paper Automating the Creative Workflow (version 1.1.8, a confidential draft). Every picture and film in this article was made in Plnty, and the motion reference is sampled from Yambo Studio's films.
- J. L. Lush. Animal Breeding Plans. Iowa State College Press, 1937.
- D. S. Falconer and T. F. C. Mackay. Introduction to Quantitative Genetics, 4th edition. Longman, 1996.
- Design Council. The Double Diamond. Framework and account of its use.
- W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3–4), 1933.
- D. A. Schön and G. Wiggins. Kinds of seeing and their functions in designing. Design Studies, 13(2), 1992.
- D. Kirsh and P. Maglio. On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 1994.
- M. A. Boden. Creativity and artificial intelligence. Artificial Intelligence, 103(1–2), 1998.
- G. Wallas. The Art of Thought. Jonathan Cape, 1926.
- J. P. Guilford. Creativity. American Psychologist, 5(9), 1950.
- D. T. Campbell. Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological Review, 67(6), 1960.
- D. K. Simonton. Creativity and discovery as blind variation: Campbell's (1960) BVSR model after the half-century mark. Review of General Psychology, 15(2), 2011.
- A. Koestler. The Act of Creation. Hutchinson, 1964.
- D. A. Schön. The Reflective Practitioner: How Professionals Think in Action. Basic Books, 1983.
- G. Goldschmidt. The dialectics of sketching. Creativity Research Journal, 4(2), 1991.
- A. Clark and D. Chalmers. The extended mind. Analysis, 58(1), 1998.
- R. E. Beaty, M. Benedek, P. J. Silvia and D. L. Schacter. Creative cognition and brain network dynamics. Trends in Cognitive Sciences, 20(2), 2016.
- M. Polanyi. The Tacit Dimension. Doubleday, 1966.
- T. M. Amabile. Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social Psychology, 43(5), 1982.
- Anthropic. The Model Context Protocol. Specification and documentation, November 2024.
- E. Cherry and C. Latulipe. Quantifying the creativity support of digital tools through the Creativity Support Index. ACM TOCHI, 21(4), 2014.
- Yambo Studio. Films and motion design, 2014 to 2026.
- ComfyUI contributors. ComfyUI. Open-source node-based environment.
- S. Yao et al. Tree of Thoughts: Deliberate problem solving with large language models. NeurIPS, 2023.
- Z. Wang et al. GenArtist: Multimodal LLM as an agent for unified image generation and editing. NeurIPS, 2024.
- J.-B. Mouret and J. Clune. Illuminating search spaces by mapping elites. arXiv, 2015.
- J. Lehman and K. O. Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation, 19(2), 2011.
- L. Zheng et al. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. NeurIPS Datasets and Benchmarks, 2023.
- L. Gao, J. Schulman and J. Hilton. Scaling laws for reward model overoptimization. ICML, 2023.
- R. A. Finke, T. B. Ward and S. M. Smith. Creative Cognition: Theory, Research, and Applications. MIT Press, 1992.
Run a skill on your own moodboard
Select a few pictures on a board, open Rothko's chat and press Run a creative skill.




























