Building SolvHive

I have been building SolvHive as a place where a simulation can outlive the laptop it was run on. This is the account I want on the blog: why I started, how the first version lived on Base44, how I brought it into Cursor and replaced the platform piece by piece, and how the parsers and the media pipeline deal with the files engineers actually export.

It matches the repository as it stands in November 2026. The short map for GitHub is the repository README. The site itself is solvhive.com.

Why I wanted this

A finished FEA or CFD study is a pile of decisions: the mesh, the contacts, the loads, the solver controls, the picture that shows whether the answer is believable. Most of that pile stays on a personal machine or a company drive. Software already has a public place to put a project, read the write-up, and take the files. Simulation did not.

I wanted one page per study. The page should show the write-up, the files, a short loop of the result, and the settings the solver actually used, read from the files rather than retyped into a form. People should be able to find a project, argue with it in a thread, and keep their own copy. The first cut is FEA and CFD only. Thermal, electromagnetics, and the rest can wait until the page for these two is honest.

The name is SolvHive. It is the hive for solvers, and for the people who run them.

The first version, on Base44

I did not start by writing an API. I started on Base44, because I wanted the product in front of me before I wanted a server of my own. Base44 gave me JSON-schema entities, sign-in, file upload, and a place to hang a few backend functions. I spent that time on the engineering product.

The shape I settled on is still the shape of the data.

A project has a title, a markdown description, a simulation type of FEA or CFD, a solver, material and domain tags, result images, a public or private flag, and later an analysis_settings object. A project file is metadata plus a URL: name, category, size, and a text preview when the file is text. Comments thread through parent_id. A vote is one row per person per project or comment. A watch follows a project. A follow connects two people. A public user profile holds the bio, the avatar, the tags, the links, and a short list of feats, so a visitor can see who wrote a comment without being handed the account table.

I wrote the interface to feel like the tools engineers already live in. A sidebar, a feed, tag pills, a markdown readme, a star and an eye. Light and dark come from CSS variables and next-themes. Action buttons run blue into green. Destructive actions run orange into red. Guests can read everything. Starring, watching, commenting, following, uploading, and editing send you through sign-in and bring you back to the same page.

I also wrote the terms before I asked anyone to upload. Public projects are public in the strong sense: someone with the link may download, copy, and adapt the files. If that is wrong for a given study, the study stays private. Signup requires the checkbox.

By the end of that stretch I had the explore feed, project pages, upload and edit, profiles, threaded comments, votes, watches, follows, zip download, a first ANSYS parser, and the terms. The reputation badge went up and came back down. I still want a trust signal. I did not want a badge I could not explain, so the votes stayed and the badge left the interface.

Base44 was the right scaffold. It was the wrong place to keep going once the files got serious. Solver decks are large, reports are messy, animations need a transcoder, and I wanted the bytes in a bucket I control. That is the point where I exported the project and opened it in Cursor.

Opening it in Cursor

Cursor is where the cleanup happened. The Base44 export was a Vite React app with the pages, the Tailwind theme, and the entity schemas under base44/entities/. The pages were written against a client called db, injected as globalThis.__B44_DB__. I kept that call shape on purpose. src/api/base44Client.js is imported first, and it installs a client that speaks my API with the same entity methods the pages already used. Rewriting every screen at the same time as replacing the server would have frozen the product.

What I removed, over the following work, was the assumption that Base44 was still the backend. vite.config.js is now plain Vite. The dev server proxies /api and /uploads to the local API on port 3001. Auth, storage, and the parser functions moved into backend/.

A few scars from the export are still visible, and I have been treating them as compatibility rather than as a second product. Token keys are still base44_access_token. Some pages still begin with the globalThis.__B44_DB__ fallback so a screen can render even if the client import order slips. The fallback returns empty lists. It is a guard, and the real client is the one that must load.

The frontend

The site is a single React 18 app in src/. Vite builds it. Tailwind and the shadcn primitives do the components. React Router keeps one layout around the public pages and gates /upload and /project/:id/edit. The homepage stays in the first download. Project, profile, auth, terms, privacy, and feedback are lazy.

I host that build on Cloudflare Pages, on solvhive.com. Pages is a static host, and it was answering /api/* with index.html, including Google's callback. functions/_middleware.js is the fix. Any path under /api or /uploads is fetched from api.solvhive.com, and redirects are passed through untouched so the Google hop can still land on the site with the session token.

In development I do not need that function. Vite proxies the same two prefixes to localhost.

The API, the database, and Render

The API is Express, in backend/. It listens on PORT, which on my machine is 3001 and on Render is whatever the platform sets. One proxy sits in front of it, so trust proxy is 1 and rate limits see the visitor address.

I host the process on Render, at api.solvhive.com. The database is Postgres on Render, through DATABASE_URL. Boot is strict about that string. A production process refuses a localhost database. A development process refuses a Render host. ALLOW_PROD_DB_LOCAL is the only override, and it stays unset, so a laptop script cannot attach to the live database by accident.

The schema is small on purpose. accounts holds email, a normalized email, the password hash, the Google subject, the username, and whether the address is verified. otps holds a hashed code, a purpose (verify or reset), an expiry, an attempt count, and the time it was sent. Everything the product calls an entity is one row in records: entity name, id, owner, timestamps, and a JSON document. Projects stay documents. Search does not get a second table to drift out of date. A project_tags function and expression indexes sit on that JSON. Migration 008_project_search.sql turns on pg_trgm and fuzzystrmatch.

The SQL files in backend/migrations/ run at startup: media rows, parse results, stored-object bookkeeping, usernames, storage locations, publish jobs, raw-media cleanup, search, and the later search updates that read a parsed solver header.

A cold Render process answers a login with a gateway status or an HTML error page. The client treats 502, 503, 504, 522, and 524 as the machine waking up, and it retries. Those statuses are not a wrong password.

Ownership is checked in the API. Public reads stay open. Creating a row requires a signed-in user. Updating or deleting a community row requires the owner or an admin. A project that is publishing or parsing rejects new uploads and edits until the job finishes. Once it is published, analysis_settings cannot be hand-edited. The way to change them is to re-parse.

The R2 layer

Solver files do not belong in Postgres, and they do not belong on the Render disk. That disk is ephemeral. I put bytes in Cloudflare R2, in two buckets that the process refuses to confuse.

solvhive-media is public. Converted images, posters, WebM, and MP4 live there with a year-long immutable cache. The public base URL is what the gallery loads.

solvhive-solved is private. Decks, logs, and reports live there. The browser uploads with a presigned multipart PUT, in 8 megabyte parts, and downloads with a signed GET that lasts fifteen minutes. The bucket needs a CORS rule that exposes ETag, or the browser cannot complete the multipart upload.

The code is backend/services/storage/r2.js, with key rules in backend/src/utils/r2Keys.js and backend/services/storage/keys.js. STORAGE_DRIVER=local is the development driver and writes under backend/uploads. Production must set STORAGE_DRIVER=r2, and it must set R2_ENDPOINT, the access keys, and a public base URL. The public URL is configuration. The API never trusts the Host header of the incoming request when it builds a file URL.

R2 rejects the AWS SDK's default flexible checksums on presigned uploads. The client sets checksum calculation and validation to WHEN_REQUIRED, so those PUTs go out without the extra checksum headers.

Older rows still pointed at localhost:3001 or at files that had been stored on the API host. scripts/migrateMediaUrls.js rewrites those onto the buckets. scripts/configureR2Lifecycle.js expires abandoned tmp/ objects. Replacing an avatar or a render unlinks the database row. I do not yet delete every previous object in the same gesture, so a replaced file can linger until lifecycle or a cleanup script catches it.

When the driver is R2, POST /api/upload returns 409 and tells the client to use the direct solved upload. The API should not be a tunnel for multi-gigabyte decks.

Email codes

Signup is email and password, then a code, then a session. I send the code with Resend, from SolvHive <verify@solvhive.com>. The message is a six-digit code and a line that it expires in ten minutes. The HTML escapes the code. The code is never put in the JSON response.

The flow I implemented:

1. POST /api/auth/register checks the address, rejects disposable domains, rejects a password under six characters, and rejects an email that already exists. The account is stored with email_verified false. A User record is created in the same step.
2. The code is six digits from a cryptographic random integer, padded. Only a bcrypt hash is stored, with purpose verify, a ten-minute expiry, and attempts at zero.
3. POST /api/auth/verify loads the row, deletes it if it is expired or already at five attempts, and on a mismatch increments the counter. Five failures delete the row. The messages say whether the code expired, locked, or is simply wrong, and how many tries remain.
4. A correct code marks the email verified and returns a JWT that lasts seven days. Login before that returns 403.
5. Resend waits sixty seconds, measured from sent_at. It is also limited by IP and by address. Asking to resend for an unknown email, or for an email that is already verified, returns the same { ok: true } so the route does not reveal which addresses exist.

If RESEND_API_KEY is empty, development prints the code in the API log and production returns 503. A send that throws becomes 502 and leaves the account in pending_verification, so the person can ask for another code instead of being told the account was never created.

Password reset uses the same table with purpose reset. An unknown email still returns success. In development the response includes a reset URL. In production it does not.

Rate limits sit on the whole /api/auth router, and again on signup, verify, and resend. I would rather lock a noisy address out for an hour than let someone walk the code space.

Google

Google is a second door into the same accounts table. I did not want a parallel user base.

The button sends the browser to GET /api/auth/google?returnTo=.... The return path has to be a relative path on my site. A protocol-relative URL, a backslash, or a token already sitting in the query is stripped. The API signs a state JWT that lives ten minutes and carries that path, then redirects to Google with openid email profile and prompt=select_account. The redirect URI is GOOGLE_REDIRECT_URI, or https://api.solvhive.com/api/auth/google/callback when that variable is unset. It has to match the Google Cloud console character for character. The Pages middleware exists so this callback is not swallowed by the static site.

On the way back I verify the state before I touch the code. A cancelled consent, a missing client secret, an expired state, or a token exchange that fails all redirect to /login with a short error. I then read the OpenID userinfo endpoint. I continue only when Google sets email_verified to true and the address passes the same validator as a typed email.

Lookup is by google_id first, then by email. If the person already has a password account, I attach the Google subject and, if they had never finished the code, I mark the email verified. Google has already verified it. If they are new, I create the account with a random password hash they will never use, email_verified true, and the Google subject stored. A unique-violation on that insert means a parallel callback won the race. I load the row that won and continue. The session cookie is not a cookie. It is the same seven-day JWT, placed on the return URL as access_token, and the client strips it into local storage.

GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET live on the API. If either is missing, that redirect lands on /login with a configuration message instead of failing halfway through the token exchange.

How a project becomes a page

Upload writes a project, then files, then media. Publish is a job, not a button that parses inside the request. backend/services/publish/worker.js claims a row, loads the solved files from local disk or from the private bucket, and runs the ANSYS bundle. A preview parse can refresh the panel while you are still editing. A sealed parse runs only while the project is publishing or parsing, and then the settings lock.

ds.dat is materialized to a temp file and streamed, because it is the large one. Reports and the other sidecars are read with a size cap. A file the classifier does not recognize is skipped and listed. A missing object is recorded as failed, and the rest of the bundle still merges.

Search is separate from that panel. GET /api/projects/search only returns projects that are public and published. A query ranks an exact title, then the first word, then a prefix, then a substring, then a small typo, then a trigram, then solver or software text, then a tag. I wanted benm to find a beam study, and I wanted ANSYS to find a project whose parsed header says ANSYS Mechanical, ANSYS Fluent, or ANSYS 2021 R1. Filters for tags, FEA or CFD, and software combine. The packages are ANSYS, COMSOL, Abaqus, and SolidWorks Simulation. Older rows that stored ansys_mechanical or ansys_fluent are read as ANSYS. When the upload form left the package empty, the filter still classifies the project from analysis_settings.summary and the solver and version lines. Facet counts ignore their own filter and respect the others, so the checkboxes stay usable while one is ticked. The notes for the query string are in docs/search.md in the repository.

The log parsers, and the faults they are built for

A solver log is a text file that was never designed as an API. My rule is that the original file stays untouched, a bad line never aborts the file, and anything I could not read remains visible as an unparsed range.

parseFile in backend/services/parsers/engine.js is the path for a single file requested from the project page.

The file is too big. Above MAX_PARSE_FILE_MB (500 by default) I return an empty result with that sentence in errors and the byte count filled in. I do not sample a random middle.

The bytes are not the encoding I hoped. The head is decoded before any plugin sees it. A BOM selects UTF-8 or UTF-16. A declared charset is tried next. Then strict UTF-8. Then windows-1252. Mechanical reports on Windows are often the last of those. I do not call Buffer.toString() on report bytes, because that hides a bad sequence instead of telling me the file was not UTF-8.

I might have the wrong solver. detect.js looks at the filename and the first 64 KB. ds.dat or a leading ANTYPE is ANSYS at 0.95. MatML, a .mntr, or the word ANSYS in the head follow. COMSOL and Abaqus match their banners and set the version from that banner. Anything else is unknown at 0.1. Each plugin then returns its own confidence. I take the highest score at or above 0.4. Under that, generic.js runs. It only emits a table when a line splits on tab, semicolon, comma, or wide spaces and at least sixty percent of the cells parse as numbers. Confidence on that path is 0.25, and metadata.fallback is true, so the UI can show that this was a guess.

A line is hostile. Lines are streamed. A line past the length cap is warned and skipped. An empty line is claimed so it does not show up as a hole. A plugin catches a bad line, warns, and continues. I will not let one corrupt increment throw away the convergence table above it.

Numbers lie in plain text. normalizeNumericToken keeps the raw spelling and a parsed value. NaN and infinities are explicit. Thousands separators (1,234.5) lose their commas. A single comma between digits (1,25) becomes a decimal point, which is how a lot of European logs are written. A Fortran exponent (1.2D-03) becomes 1.2e-03 before Number sees it. If it still is not finite, the cell stays text. The ANSYS quantity splitter is stricter. It takes a leading number and a unit from strings like 4.1819e-005 m, and it refuses to treat a word such as a release name as a unit.

I have to show my work. Every claimed line number goes into a set. After the plugin returns, consecutive unclaimed lines become one range with a short text preview, capped at 2000 characters. That range is both unparsedRanges and raw_sections. Completeness is how much of the file was claimed, not how confident the detector felt.

The work must not stall the API. The parse runs in a worker thread. PARSE_TIMEOUT_SECONDS kills it. The result is inserted as a new parse_results row with the parser version, so an older reading of the same file is still there. One table exports as CSV or JSON without reparsing.

A model must not invent a stress. ENABLE_LLM_PARSE defaults to false, and no model runs inside the API process. If I turn it on, it only runs when the file is unknown or completeness is under 0.5. Every numeric token in an accepted row or scalar has to occur verbatim in the first 64 KB of the file. A miss drops that row and adds a warning. The result is capped at 0.35 confidence and tagged source.origin: "llm". I would rather show a gap than a modulus the log never contained.

Adding a solver is one file in backend/services/parsers/plugins/. Export id, canParse({ filename, head, detect }), and parse(stream, options). Record rows through options.context (scalar, row, claim, warn). Do not throw on a bad line. The registry loads the directory and sorts generic last. That is the door I left open for OpenFOAM, LS-DYNA, Fluent, and Nastran. I have not walked through it yet. The three plugins that are real today are ANSYS, COMSOL, and Abaqus, and they pull solver version, materials, increments, and the end-of-run marker when the log carries them.

The ANSYS bundle, and the faults in that merge

The log plugin answers "what tables are in this one file?" The project page asks a harder question: "what was this Mechanical study?" A study is several files that disagree in small ways. parseAnsysBundle classifies each name and merges them.

Mesh blocks will eat the parser. A ds.dat is mostly nodes and elements. parseDsDat streams the file and, on nblock, eblock, or cmblock, discards lines until the terminating -1. I still read an element count off the eblock header when it is there. Bodies, contacts, and loads are the comments and commands around those blocks. The cap for a deck read into memory, when I cannot stream, is 80 MB. A report cap is 20 MB. Over the cap, that file is failed and the others continue.

The report is a MIME bundle, and many of its charts are empty. An MHT file carries an HTML part and then PNG parts in base64. I split on the MIME boundaries, decode the HTML with the same charset ladder, and lift each PNG by its Content-Location. Result charts in a Mechanical report are often white axes with nothing plotted. isBlankChart resizes the PNG to 64 by 64 and counts pixels above 245 in every channel. At or above 97 percent white, or if the image is under 120 by 80, I drop it. A force plot or a body contour uses more of the frame and stays.

The same fact appears three times. mergeAnsys prefers a present value over Unspecified, and it keeps a chain so a missing report does not wipe a value the deck already had. Version is the report product plus the solve.out build when both exist. Status is "Converged", "Converged with warnings", or "Not converged" from the solve file, and only then from the report's status line. Unit system prefers the report, then .cnd, then the deck. Project title skips the literal word "Project". Environment temperature prefers the report, then the deck's reference temperature.

"Program Controlled" is a choice, and it is kept as a tag. When the report says program controlled and another source has the value the program actually used, I keep the resolved value and tag it Program Controlled. When nobody resolved it, the words stay as text.

A missing field stays missing. unspecified() is the value Unspecified. Downstream code checks isUnspecified before it prefers the next source. An empty string is treated the same way. The panel can render that absence on purpose.

Geometry and mesh numbers get mixed up. Node and element counts, and the mesh metric, are lifted out of the report's mesh group into their own stats. The group that remains is the controls, with those labels removed, so the same number is not printed twice under two headings.

Loads need a figure and a curve. A load from the report is matched to a CAERep point list by name, then to a deck component, then to a figure whose target is that load. If the points exist, the component list is cleared so I do not draw both. Results get the same name match for their contour.

The report was never exported. Mesh, results, and scope carry a hint string when the report is absent, and the analysis block falls back to end time, substeps, solver, tolerance, and restart from ds.dat. The page can say what it is missing. The schema version on the object is 2. leafPaths walks every field that has a value, and it skips deck and raw, which is how I check that a change still covers the tree.

Publish and parse can race. While publish_state is publishing or parsing, uploads and edits return 409. A job whose project has already left that state is skipped as stale, so a slow worker cannot overwrite a newer seal. After publication, a direct write to analysis_settings is rejected. Re-parse is the supported door.

Animations and the other media faults

People were uploading result movies that were minutes long, and GIFs that were a single deformation cycle buried in a hundred megabytes. I wanted the card on the explore page to move, and I wanted it to be small.

POST /api/media checks the file before it stores a row. The response comes back with status: "pending". Conversion runs on a queue, CONVERSION_CONCURRENCY at a time. REDIS_URL selects BullMQ. Without it, the queue is in-process. GET /api/media/:id is the owner, or an admin, and reports pending, processing, done, or failed.

The bytes must match the name. file-type reads the content. JPEG, PNG, WebP, GIF, AVIF, APNG, MP4, MOV, WebM, and MKV are the allowed set. A claimed extension that disagrees is TYPE_MISMATCH. An empty file is EMPTY_FILE. Over the image or video megabyte cap is 413. A solver log posted to this route is told to use the solved upload. An image posted to the solved route is told to use /api/media.

A still can be a lie. sharp decodes with a pixel cap, failOn: 'error', and auto-rotate, then writes a clean file so orientation and metadata do not survive. A pixel-limit error becomes DIMENSIONS_TOO_LARGE. Anything else from the decoder is CORRUPT_MEDIA. Output is WebP at the configured quality. Widths 400, 800, and 1600 are written only when the source is wider than that width. A 200 pixel thumbnail follows the same rule. I never upscale.

A GIF is a video. If sharp reports more than one frame, or the file is APNG, I probe it with ffprobe and send it down the motion path. A single-frame GIF stays a still.

The loop should bounce. Deformation clips look wrong if they cut and jump back to the first frame. encodeVariant takes the first MAX_SOURCE_SECONDS of the source and runs a filter: drop to MOTION_FPS, scale to the target width without upscaling, split, reverse the second copy, and concatenate. The card target is 480 pixels. The full target is 1080. Each becomes an H.264 MP4 with +faststart, a VP9 WebM, and a WebP poster from the first frame. The gallery component plays the WebM muted, inline, and looping, and offers the MP4 as the second source.

Reverse will blow up on a 4K frame. ffmpeg's reverse filter holds the clip in memory. fitMotionWidth computes the scaled frame and, if the pixel count would pass MAX_REVERSE_FRAME_PIXELS (921600, a 720p frame), shrinks until it fits. A frame that still cannot fit throws, and that media row fails instead of taking the worker down.

VP9 sometimes rejects the filter graph. The WebM encode is tried with the ping-pong graph. On failure I delete the partial file and encode again with fps and scale only, no reverse. The MP4 has already been written with the ping-pong graph. The card still moves. The WebM is the one that degrades, and only that variant.

ffmpeg can hang. Each spawn has FFMPEG_TIMEOUT_SECONDS. The child is killed, and the error kept for the row is the last 500 characters of stderr. The temp directory is removed in a finally block whether the encode worked or not.

Originals. KEEP_ORIGINALS is false in the environment I ship. The converted derivatives are what the site serves. Turning the flag on stores original.<ext> next to them. I default to off because the private bucket is for solver files, and a 100 MB source movie does not need a second life beside an 8 MB loop.

The player is src/components/LoopingMedia.jsx. A URL that is not a WebM stays an image. A WebM gets a poster, either the one the API stored or the sibling poster.webp inferred from a play.webm name.

What I watch for when something is wrong

I have stopped treating a blank panel as a mystery and started treating it as one of these cases.

If the analysis panel is empty, I look at the file list on the project first. A PDF was rejected on purpose. A ds.dat over the cap failed and should say so on that file's status. A report that never exported leaves the hint on mesh and results, with deck values in the analysis fallback. A figure that is missing was often the white-chart filter, and I would rather lose a blank PNG than show one.

If a log parse looks thin, I open unparsedRanges and the warnings. A low confidence and metadata.fallback means the generic detector ran. A warning about a truncated line or a stopped read means the cap, not a logic error in the plugin. Parser version on the row tells me whether I am looking at a result from before a plugin change.

If an animation never leaves pending, the queue is either waiting on concurrency or the worker threw. failed keeps the ffmpeg tail. The usual causes are a frame that could not be reversed, a timeout, or a file whose probe reported no video stream.

If Google dumps the person on the homepage HTML, the callback hit Pages instead of the API. The middleware and the redirect URI are the two places that fix that. If they land on /login with an error, the state expired, they cancelled, or Google did not mark the email verified.

If a signup never receives a code, production with a missing Resend key returns 503, and a Resend failure returns 502 with pending_verification. The account exists. Resend is the next step, after the cooldown.

If a file URL still points at localhost or at the API host, it predates the R2 migration. The migrate script is the repair. New uploads take the public media URL or a signed solved URL and do not go through the API disk.

Where I am

The site is up. The explore feed, the project page, profiles, comments, votes, watches, follows, terms, privacy, and feedback are the product I started on Base44, now served from my own API. Email codes and Google both create the same kind of account. Media converts. The ANSYS panel fills in from a real export. Search survives a typo. Private solver files sit in a bucket the API disk does not have to hold.

I am not done. The plugin door is open and only three solvers walk through it. Domains beyond FEA and CFD are still a tag I refused to add until the page can tell their settings apart. Project history, in the sense of a study that changes and keeps the previous reading, is not built. The reputation badge is still off. LLM parsing stays off until I am willing to defend every number it would add. Replaced objects in R2 are not always deleted in the same request as the row that pointed at them.

That is the honest state of SolvHive. The next file I care about is the next solver that shows up in an upload and currently falls through to the generic table detector.

Back to the blog