eval/stage2b/report.md

# Atlas-Vision evaluation

## Scores (stage 1 → stage 2)

| Metric | Stage 1 | Stage 2 |
|---|---|---|
| Held-out instruct loss (lower is better) | 2.2521 | 1.1602 |
| POPE random: accuracy / F1 / yes-ratio | 0.4997 / 0.2003 / 0.1257 | 0.822 / 0.8318 / 0.558 |
| POPE popular: accuracy / F1 / yes-ratio | 0.5027 / 0.2013 / 0.1227 | 0.7757 / 0.7969 / 0.6043 |
| POPE adversarial: accuracy / F1 / yes-ratio | 0.5023 / 0.2012 / 0.123 | 0.752 / 0.78 / 0.6273 |

## Detailed descriptions

**COCO_val2014_000000310196**

- Stage 1: a skier in the snow on a mountain
- Stage 2: The image features a snow-covered slope with a skier in the center, wearing red and black clothing. The skier is skillfully navigating down the hill, making their way through the snow. 

In addition to the main skier, there are several other people scattered across the scene, some of them closer to the top of the slope while others are further away or near the bottom. These individuals may also be skiing or enjoying the snowy landscape.

**COCO_val2014_000000210789**

- Stage 1: a woman and her child in the rain
- Stage 2: The image features a woman and a young girl standing in the rain, both holding umbrellas to protect themselves from getting wet. The little girl is wearing a pink dress, while the woman appears to be her mother. They are positioned close together, with the child on the left side of the frame and the woman on the right.

In addition to the two main subjects, there are several other people visible in the background, some of whom may also be holding umbrellas or trying to stay dry during the rainy weather. A car can be seen parked further back in the scene, likely waiting for its occupants

**COCO_val2014_000000429109**

- Stage 1: a bus and several other vehicles parked in front of a building
- Stage 2: The image features a busy street with several buses and cars parked or driving along the road. There are three buses in total, one on the left side of the scene, another near the center, and the third bus further to the right. A car is also visible on the left side of the image.

In addition to the vehicles, there are multiple people walking around the area. Some individuals can be seen closer to the buses, while others are scattered throughout the scene. The presence of both vehicles and pedestrians suggests that this could be a popular transportation hub or a bustling city street.

**COCO_val2014_000000211674**

- Stage 1: a bus with a red and white logo on it, carrying passengers
- Stage 2: The image features a red double-decker bus with an advertisement for "Ruta Arzola" on its side. The bus is driving down the street, and there are several people visible in various positions around it. Some of these individuals can be seen standing near the front of the bus, while others are scattered along the sides or further back.

In addition to the bus, there are two cars present in the scene – one located behind the bus and another positioned slightly ahead of it. A bicycle can also be spotted close to the right edge of the image.

## Questions in Nigerian languages (stage 2)

- **igbo** — Kedu ihe dị na foto a?
  → Na foto a, e nwere skier na snow.
- **yoruba** — Kí ni ó wà nínú àwòrán yìí?
  → Nínú àwòrán yìí, ó wà skier tí ń ski lórí òkè.
- **hausa** — Me ke cikin wannan hoton?
  → In the image, there is a skier wearing red and black clothing who is skiing down a snow-covered slope.

## Text-only check (no image)

Question: Kedu ihe bụ positron? Kọwaa ya n'asụsụ Igbo.

- Base N-ATLaS: Positron bụ eletrọn dị mma, ma ọ bụ antiparticle nke eletrọn. Ọ na-emegide eletrọn nkịtị site n'ịnwe mpempe ojii nwere otu ụgwọ ahụ mana ọdịiche dị na ntụziaka nke ụgwọ ha. Positrons na-ekere òkè dị mkpa na physics nuklia, ebe a na-emepụta ha mgbe ụfọdụ n'oge us
- With stage-2 LoRA: Positron bụ ụdị subatomic dị ka electron mana ọ nwere nnukwu ọdịiche - ọ nwere ụgwọ dị mma, ebe electron nwere ụgwọ na-adịghị mma. A na-akpọkwa positron antiparticle nke electron, n'ihi na ọ na-emegide njirimara niile nke electron ma nwee ibu yiri ya. Nchọpụta nke positrons mere site n'aka physicist Paul Dirac