Project Management for AI Projects

Two different things, both called "AI"

Worth separating clearly, because the two get conflated constantly: this page is about managing a team that builds AI systems, data preparation, training runs, evaluation, deployment. It is not about ShipSprint's own AI features, which is a separate story about connecting Claude and ChatGPT to your workspace so you can query and update it in plain language. That's a convenience for any team using ShipSprint. It has nothing to do with whether ShipSprint can track your model training pipeline, which is what this page actually answers.

And to be direct about what ShipSprint is not: it is not an ML-ops platform. There's no experiment tracking, no model registry, no integration with training infrastructure. If you need to compare hyperparameter runs or version model artifacts, that's a different category of tool and you should keep using it. What ShipSprint does is plan and track the human workflow around that work: who's doing what, is the eval milestone at risk, what's blocked and why. It's the project management layer sitting above the ML infrastructure, not inside it.

Why the human workflow still needs its own tracking

Teams building AI systems often assume that because the technical work is tracked somewhere, a notebook, an experiment log, a training dashboard, the project itself is tracked too. It usually isn't. Someone still needs to know that the labeling contractor is two days behind, that the person who owns the evaluation harness is out this week, or that a deployment is waiting on a security review that hasn't been scheduled. None of that lives in a training log, and all of it can quietly determine whether a milestone lands on time.

That gap tends to show up as a familiar surprise: the model itself was ready weeks ago, but the project wasn't, because the surrounding coordination, data access, review, deployment sign-off, was never anyone's tracked responsibility.

Iterative and uncertain, with its own stages

Building an AI system shares research's basic problem, you don't fully know how long training or evaluation will take until you've done a few rounds of it, but it has its own recognizable stages: data preparation and labeling, training runs, evaluation against a benchmark or held-out set, deployment. A board built around those stages tracks reality better than a generic backlog, and forecasting from velocity works the same way it does for research: it tells you the team's pace against its current plan, not whether the model will hit the target accuracy. No tool can promise that second thing, and ShipSprint doesn't try to.

What ShipSprint does and doesn't do here

• Tracks who's doing what and whether milestones are at risk, does not track experiments, hyperparameters or model versions
• Forecasts team pace from velocity, does not forecast model accuracy or whether training will converge
• Gives training runs and evaluation stages their own board columns, does not integrate with training infrastructure or a model registry
• Connects to Claude and ChatGPT so the workspace can be queried in plain language, a convenience layer on top of project data, not a feature of the AI system you're building
• Keeps the coordination work, data access, review sign-off, deployment scheduling, visible even when the technical work is tracked somewhere else entirely

The eval milestone, specifically

Of all the stages in building an AI system, the evaluation milestone tends to be the one that most needs project-management discipline and least gets it. Training has its own tooling and its own visible progress bars. Evaluation is often looser: a set of criteria that were agreed verbally, a benchmark someone's supposed to be running, a bar for "good enough" that lives in someone's head rather than written down anywhere. That looseness is exactly where a milestone quietly slips, not because the eval work is hard, but because nobody was tracking whether it had actually started.

Putting the evaluation stage on the board as its own set of tasks, with an owner, a WIP limit, and a place to log what the results actually were, doesn't make the evaluation itself more rigorous. It makes it visible enough that "we haven't started evaluating yet" surfaces as a fact on a Tuesday, not as a surprise the week the model was supposed to ship. That's really the whole pitch for this page: ShipSprint won't make your model better, but it will make sure nobody discovers evaluation never started until it's too late to fix.