https://editor.p5js.org/wallismb/full/QJLXuSiL-

I initially attempted to run a Node Puppeteer script locally to scrape content from MoMA's 'Art Terms' page and convert it to JSON, but encountered several errors. I decided to postpone this approach for later.

Instead, I found a Chrome extension that allowed me to download the page content directly as JSON. As with any data source, the output required cleaning and reformatting to work with my p5.js sketch. I used a Node script to restructure the data into my preferred format.

{
  "entries": [
    {
      "term": "Abstract Expressionism",
      "definition": "The dominant artistic movement in the 1940s and 1950s, Abstract Expressionism was the first to place New York City at the forefront of international modern"
    },
    {
      "term": "Acrylic paint",
      "definition": "A fast-drying paint made of pigment suspended in acrylic polymer emulsion. A key difference between acrylic paint and oil paint is that acrylics are water-based"
    },
    ...
  ]
}

code to reformat the JSON file

const fs = require('fs');
const path = '/Users/wallismillar-blanchaer/Desktop/artist-statements/puppeteer-scraper/moma-2024-10-29.json';
const outputPath = '/Users/wallismillar-blanchaer/Desktop/artist-statements/puppeteer-scraper/moma_reformatted.json';

// Read the JSON file
fs.readFile(path, 'utf8', (err, data) => {
  if (err) {
    console.error("Error reading the file:", err);
    return;
  }

  try {
    // Parse the original JSON data
    const originalData = JSON.parse(data);
    const reformattedData = { entries: [] };

    // Loop over each entry in the original data
    originalData.forEach(entry => {
      const termKeys = Object.keys(entry).filter(key => key.includes("layout/block"));
      const definitionKeys = Object.keys(entry).filter(key => key.includes("$typography/scale:down"));

      // Match terms with their corresponding definitions
      termKeys.forEach((termKey, index) => {
        const term = entry[termKey];
        const definition = entry[definitionKeys[index]];
        
        if (term && definition) {
          reformattedData.entries.push({
            term: term,
            definition: definition
          });
        }
      });
    });

    // Write the reformatted data to a new JSON file in the same directory
    fs.writeFile(outputPath, JSON.stringify(reformattedData, null, 2), 'utf8', (writeErr) => {
      if (writeErr) {
        console.error("Error writing the file:", writeErr);
      } else {
        console.log(`File successfully reformatted and saved as 'moma_reformatted.json' at ${outputPath}`);
      }
    });
  } catch (parseError) {
    console.error("Error parsing the JSON data:", parseError);
  }
});

I wanted to feed it into one of the examples that Dan shared, so I adjust the code in sketch.js a bit for my intended output.

First result — not quite

Adjustments

• In order to produce more recognizable results, I increased the order of the N-grams from “markov = new MarkovGenerator(4, 100);” to “ markov = new MarkovGenerator(6, 150);”
• I adjust the generator to also start with a known term from my JSON file, so it would begin the text with a meaningful term
• Added a formatted output using ${term}: ${description} to create a consistent structure

V2 Results — warmer

Adjustments

• Remove extra spaces, returns / breaks, brackets without corresponding closing bracket,
• Ensure each description for a term has two sentences which both end with a period