The Amazon earthworks model learned my agent's guess

A lot of Punkjazz is me testing how far a non-developer can push research with AI agents, open data, and local models, while still knowing when to stop and question the result.

The Giza radar project worked because the evaluation stayed outside the agent. The agent could help with code, comparisons, papers, and variations, while the central question remained fixed. The Göbekli Tepe work followed a similar pattern.

Amazon earthworks looked like the next useful test. The data was public, the target was visual, and the compute was ready. I wanted my AI agent to pull satellite imagery, prepare the dataset, train models, read the logs, and suggest the next move. I wanted the research loop to become more autonomous.

It failed in a useful way. My AI agent filled in missing evidence with a plausible shortcut, and I let that shortcut become the thing the model had to learn. After that, the pipeline could look productive while drifting away from archaeology.

That is the point for anyone using agents for research: the cleaner and more dangerous failure is a fast loop built around an unchecked target.

The archaeological starting point came from Peripato et al. (2023), the Science paper arguing that more than 10,000 pre-Columbian earthworks may still be hidden across Amazonia. My working set had 937 known sites.

I wanted to train a segmentation model, which means asking a model to color the pixels that belong to an object. Here it would color pixels that looked like earthworks.

That requires masks. A mask is basically an answer key: white pixels mean target, black pixels mean background. I had coordinates, which only say where a site was recorded. I did not have measured boundary shapes.

My AI agent solved that missing piece by drawing circles around the coordinates, using guessed radii between 50 m and 250 m, then turning those circles into masks. The imagery came from Sentinel-2 satellite tiles at 10 m resolution, so each pixel covers a 10 m by 10 m square on the ground.

Based on those constructed masks, only about 0.01 percent of pixels were positive, roughly one target pixel for every ten thousand background pixels. That made the task hard, but the more important problem had already entered the system. The circle size and shape were guesses. A circle can be fine as a search hint or a visual aid. Once it becomes the answer key, the model starts learning the guess.

Before the segmentation runs, I had a classifier branch that produced candidate detections. According to my chat notes, one version produced 12,681 candidates above a confidence threshold of 0.65. The exact count is directional, but the workflow mistake is clear enough.

I did some manual checking in Google Earth early on. Several confident candidates were sitting on empty land, trees, ordinary canopy texture, or places where I could not see anything plausibly archaeological. I flagged this to my AI agent. The agent responded like a useful coding agent: it made a new version.

At the time, that felt like a hard stop of sorts. I had seen the bad targets, told the agent, and pushed the loop to correct itself. The proper stop would have been much stricter. I should have paused the whole project, sampled candidates by hand, audited the labels, and decided whether the task still made sense. Instead, I treated the next version as another step in the autonomous loop.

That small difference matters: telling an agent that something looks wrong can produce better motion without producing better evidence.

The important question was simple: are the things being flagged actually things?

The best-looking segmentation run was v5. It trained for 150 epochs and used 5-fold cross-validation, where the data is split into five parts and tested five ways.

The headline number was F1 = 0.9100 at epoch 28. F1 is a score that balances false alarms and misses. With verified labels, 0.91 would look excellent. In this case it measured how well the model matched masks generated from guessed circles.

The model improved when it traced the circular target around a known coordinate. Real earthworks can be irregular, eroded, partial, hidden under canopy, or hard to separate from ordinary landscape texture. The training target looked clean because my AI agent had made it clean.

A low score would have forced a rethink. A high score made the wrong path feel alive.

After v5, the agent tried to deal with the rarity of target pixels. One run used copy-paste augmentation, which means taking positive regions from one training example and pasting them into another so the model sees more examples.

In a normal dataset, that can be useful. Here the pasted regions came from guessed circles. The model could now react to artificial fragments, seams, color differences, texture breaks, or other traces created by the augmentation process.

The logs show the shape of the failure. In the heavy copy-paste run, the model called far too many pixels positive. The training loss improved, which normally feels like progress, but the overlap score stayed at zero. In simple terms, the model was becoming more confident without drawing the right thing.

The next run was revised. The preserved log still contains copy-paste entries, so I cannot describe it as a clean no-paste run. What changed is that the behaviour collapsed: by the second epoch, the model was almost never calling any pixel positive, and the overlap score was still zero.

That swing is the useful signal. When one version sees targets everywhere and the next version almost never sees them, the pipeline is probably reacting to the training setup.

The agent also ran a one-off diagnostic between these versions that separated pasted validation patches from natural validation patches. It reportedly scored well on pasted patches and zero on natural ones. That output stayed in a chat window, so I keep it as context rather than evidence. The preserved logs already carry enough of the story.

By this point, the agent was changing losses, fine-tuning checkpoints, adjusting augmentation, and trying nearby variants around the same weak target. From the outside, it looked like research: logs, charts, decisions, next runs. From the inside, the work had become motion.

A later classifier branch produced another impressive score after adding elevation data. That also became suspect once I looked at the validation split. The positive examples were geographically clustered, so random cross-validation could put nearby patches from the same satellite tile into both training and validation. A model can learn the local look of a tile and seem to generalize.

I did not rerun that branch with a strict tile-level split, so I treat that diagnosis as moderate confidence. The broader lesson is that validation design can make a model look smarter than it is.

This connects to the other Punkjazz experiments. In Giza and Göbekli, the comparison target stayed outside the loop. In the DS4 Metal-to-CUDA experiment, cheap internal checks missed important unfinished pieces. The Amazon project sits in the same family. My AI agent helped create the target, the model learned that target, and the metrics treated the fit as progress.

The practical rule I trust more now is simple: autonomy works best after the evaluation target is fixed, external, and hard to fake. When the agent is still shaping the target, I have to stay close.

The next version of this workflow needs gates before long runs.

First, a label-lineage gate. Every target should answer where it came from and what physical thing it claims to measure. A coordinate, a guessed circle, a polygon, and a lidar-derived boundary are different kinds of evidence.

Second, a manual-inspection gate. Before architecture work begins, I want a small Google Earth or GIS review of high-confidence outputs. If the top candidates are empty land, trees, or ordinary texture, the project stops or changes formulation.

Third, an evaluation-design gate. Remote-sensing projects with clustered positives need geographic separation between training and validation. Nearby patches are too easy for a model to exploit.

Fourth, a diagnostic-artifact gate. Every one-off script should write timestamped outputs to disk. Chat output is useful while working, but fragile as evidence.

With point coordinates only, I would begin with candidate detection, density estimation, or weak supervision. Supervised pixel segmentation should wait for measured polygons or lidar-derived boundaries. If I use circles again, they will stay clearly marked as guesses throughout the pipeline.

The agent can still pull imagery, write data loaders, run experiments, summarize logs, and propose next steps. The constraint is simple: a convenience assumption cannot silently become ground truth.

A serious second attempt would start with the data. For supervised segmentation, the project needs real boundary annotations, likely from lidar-derived archaeology datasets. Better imagery would also help. Sentinel-2 has 10 m bands, so a 50 m feature spans only about 5 pixels. NICFI Planet basemaps provide 4.77 m imagery for tropical regions.

The order should be labels, validation design, imagery, and only then architecture.

• Peripato et al. (2023), More than 10,000 pre-Columbian earthworks are still hidden throughout Amazonia, Science.
• de Souza et al. (2018), Pre-Columbian earth-builders settled along the entire southern rim of the Amazon, Nature Communications.
• Prümers et al. (2022), Lidar reveals pre-Hispanic low-density urbanism in the Bolivian Amazon, Nature.
• Planet, NICFI Satellite Data Program User Guide.