AGI Strategy - Day 2

Detailed Instructions for Day 2 Link to heading

Goal: Load the ToxiGen dataset, understand its structure, and create a balanced “golden set” of data to use for testing later.

Step 1: Environment Setup

  • You will need the datasets library from Hugging Face.
  • Run: pip install datasets pandas
  • (Optional) If you want to see the data in a table format easily, pip install jupyter and use a notebook, or just use standard Python scripts.

Step 2: Load and Inspect (The “Deep Dive”)

  • Use the code checkpoint you provided.
  • Crucial Step: The ToxiGen dataset on Hugging Face is quite large. When you load it, look specifically for the columns that indicate:
    • The Text: The actual hate speech or benign text.
    • The Group: Which minority group is being targeted?
    • The Label: Is it toxic (1) or benign (0)?
  • Note: The ToxiGen data might have separate configs. If load_dataset("skg/toxigen-data", "train") throws an error about missing configs, try load_dataset("skg/toxigen-data", "annotated") as that usually contains the human-validated labels which are better for evaluation.

Step 3: Create the Stratified Subset

  • “Stratified” means ensuring your sample has an equal representation of groups.
  • The Logic: You don’t want 900 examples about one group and only 100 about others. You want a balanced mix.
  • I have provided code below to help you do this easily using Pandas.

Step 4: Save Your Work

  • Once you have your 500-1000 rows, save them to a file (e.g., toxigen_subset.csv) so you don’t have to re-process the huge dataset every time you run your code.