Building Mend-Ai: A Canvas That Draws Diagrams From Plain English

A few weeks ago I wanted a simple whiteboard app I could sketch flowcharts on. Somewhere along the way that turned into something bigger: a canvas where you can either draw shapes by hand, or just type “draw a login flow with three states” and watch it appear, fully editable. That project became Mend-Ai, and you can try the live version at Mend-Draw.

Here’s what I learned building it.

The core idea: two ways to build the same thing

The whole app is built around one principle — manual mode and AI mode should produce the exact same kind of object. A shape you drag onto the canvas and a shape the AI generates need to be indistinguishable once they exist. Otherwise you end up with two separate systems: “real” shapes you can edit, and “AI” shapes that are just a picture bolted on top.

So early on I settled on a plain, serializable shape schema — something like:

{
  id: "shape_182",
  type: "rectangle", // or 'ellipse', 'diamond', 'arrow'...
  x: 120,
  y: 80,
  width: 160,
  height: 80,
  label: "User Login",
  color: "#4f46e5"
}

Every shape, whether hand-placed or AI-generated, is just an object in this format sitting in an array. The canvas renderer doesn’t care where it came from — it just draws whatever’s in the array. That decision made everything downstream (editing, exporting, saving) so much simpler than I expected.

Wiring up natural language generation

For the AI side, I used Groq mainly because of the speed — generating a diagram feels instant instead of the multi-second wait you’d get with some other providers, which matters a lot for something meant to feel like a canvas tool and not a “submit and wait” form.

The trick isn’t really prompting the model to “draw” anything — it’s prompting it to return the same shape schema as JSON, which the canvas already knows how to render:

const response = await fetch("https://api.groq.com/openai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${GROQ_API_KEY}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "llama-3.1-70b-versatile",
    messages: [
      {
        role: "system",
        content: "You output ONLY a JSON array of shape objects matching this schema: {id, type, x, y, width, height, label, color}. No prose, no markdown fences."
      },
      { role: "user", content: userPrompt }
    ]
  })
});

Getting the model to reliably return clean JSON — no stray commentary, no markdown code fences — took more iteration than the actual canvas rendering did. Being explicit in the system prompt about the exact shape of the output, and adding a defensive strip-and-parse step on my end (in case a stray ```json fence sneaks in anyway), saved me from a lot of silent failures.

The “Improve Selected” feature works the same way, just with more context: I pass in the currently selected shapes as JSON along with the user’s instruction, and ask the model to return an updated version of only those shapes. Same schema in, same schema out — no special case needed.

Canvas rendering without a framework fighting you

I didn’t reach for a heavy diagramming library — partly to keep bundle size down, partly because I wanted full control over how shapes connect and resize. That meant handling the boring-but-important parts myself: drag detection, resize handles, snapping arrows to shape edges when you move something. None of it is glamorous, but it’s the difference between a demo and a tool people actually want to use.

One small thing that made a disproportionate difference: recalculating arrow endpoints whenever a connected shape moves, instead of storing arrow positions statically. Otherwise your diagram falls apart the moment you drag one box.

Persistence and export

Since diagrams are just arrays of plain objects, saving them is almost free — I used Supabase to store diagrams as JSON blobs per user, and exporting to PNG/SVG follows the same pattern: walk the shape array, draw it to an offscreen canvas or build the SVG string, then trigger a download. No extra “export format” logic needed beyond the renderer I already had.

What I’d do differently

If I started over, I’d build the shape schema and a small validation layer before touching the AI integration — I ended up retrofitting validation after a few bad responses slipped through and broke rendering. Cheap lesson, in hindsight.

If you want to see the whole thing end to end, the repo has setup instructions, or you can just try it live — no login needed to play around in manual mode.