Sync third-solution for HF Jobs Docker image
Browse files
README.md
CHANGED
|
@@ -1,260 +1,13 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
The task of the competition was "to create an AI that can solve reasoning tasks it has never seen before."
|
| 15 |
-
|
| 16 |
-
This competition is highly specific and not similar to any other DS competitions I have participated in. This particular repository and the description below are addressed to those who participated in the competition or at least familiar with the task. I recommend you to read the information [on the official Kaggle page](https://www.kaggle.com/c/abstraction-and-reasoning-challenge), [github repository](https://github.com/fchollet/ARC) and / or [the original article about abstraction and reasoning](https://arxiv.org/abs/1911.01547)
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
# 2. My approach
|
| 20 |
-
|
| 21 |
-
In general, my method can be described as a domain-specific language (DSL) based. I have created the abstract representation of the colors, image blocks extracted from the original input image, and different binary masks. Using these abstractions, I wrote several "predictors" that try to derive a set of transformations used to get every output image from the corresponding input image and, if it succeeds – apply these transformations to test input to get the final answer.
|
| 22 |
-
|
| 23 |
-
## 2.1 Abstractions
|
| 24 |
-
Each abstract representation of color, block, or mask is a JSON-like object that describes a set of transformations used to get this object from the original image. Below are several examples.
|
| 25 |
-
|
| 26 |
-
### 2.1.1 Colors
|
| 27 |
-
I use a few ways to represent colors; below are some of them:
|
| 28 |
-
|
| 29 |
-
- Absolute values. Each color is described as a number from 0 to 9.
|
| 30 |
-
Representation: `{"type”: "abs”, "k”: 0}`
|
| 31 |
-
|
| 32 |
-
- The numerical order of color in the list of all colors presented in the input image sorted (ascending or descending) by the number of pixels with these colors.
|
| 33 |
-
Representation: `{"type”: "min”, "k”: 0}`, `{"type”: "max”, "k”: 0}`
|
| 34 |
-
|
| 35 |
-
- The color of the grid if there is one on the input image.
|
| 36 |
-
Representation: `{"type”: "grid_color”}`
|
| 37 |
-
|
| 38 |
-
- The unique color in one of the image parts (top, bottom, left, or right part; corners, and so on).
|
| 39 |
-
Representation: `{"type": "unique", "side": "right"}`, `{"type": "unique", "side": "tl"}`, `{"type": "unique", "side": "any"}`
|
| 40 |
-
|
| 41 |
-
- No background color for cases where every input has only two colors and 0 is one of them for every image.
|
| 42 |
-
Representation: `{"type": "non_zero"}`
|
| 43 |
-
|
| 44 |
-
Etc.
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
### 2.1.2 Blocks
|
| 48 |
-
Block is a 10-color image somehow derived from the input image. Each block is represented as a list of dicts; each dict describes some transformation of an image. One should apply all these transformations to the input image in the order they are presented in the list to get the block. Below are some examples.
|
| 49 |
-
|
| 50 |
-
#### The first order blocks (generated directly from the original image):
|
| 51 |
-
|
| 52 |
-
- The image itself.
|
| 53 |
-
Representation: `[{"type": "original"}]`
|
| 54 |
-
|
| 55 |
-
- One of the halves of the original image.
|
| 56 |
-
Representation: `[{"type": "half", "side": "t"}]`, `[{"type": "half", "side": "b"}]`, `[{"type": "half", "side": "long1"}]`
|
| 57 |
-
|
| 58 |
-
- The largest connected block excluding the background.
|
| 59 |
-
Representation: `[{"type": "max_block", "full": 0}]`
|
| 60 |
-
|
| 61 |
-
- The smallest possible rectangle that covers all pixels of a particular color.
|
| 62 |
-
Representation: `[{"type": "color_max", "color": color_dict}]`
|
| 63 |
-
color_dict – means here can be any abstract representation of color, described in 2.1.1.
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
- Grid cell.
|
| 67 |
-
Representation: `[{"type": "grid", "grid_size": [4,4],"cell": [3, 1],"frame": True}]`
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
- The pixel with particular coordinates.
|
| 71 |
-
Representation: `[{"type": "pixel", "i": 1, "j": 4}]`
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
Etc.
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
#### The second-order blocks – generated by applying some additional transformations to the other blocks:
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
- Rotation.
|
| 81 |
-
Representation: `[source_block ,{"type": "rotation", "k": 2}]`
|
| 82 |
-
`source_block` _means that there can be one or several dictionaries, used to generate some source block from the original input image, then the rotation is applied to this source block_
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
- Transposing.
|
| 86 |
-
Representation: `[source_block ,{"type": "transpose"}]`
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
- Edge cutting.
|
| 90 |
-
Representation: `[source_block ,{"type": "cut_edge", "l": 0, "r": 1, "t": 1, "b": 0}]`
|
| 91 |
-
_In this example, we cut off 1 pixel from the left and one pixel from the top of the image._
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
- Resizing image with some scale factor.
|
| 95 |
-
Representation: `[source_block , {"type": "resize", "scale": 2}]`, `[source_block , {"type": "resize", "scale": 1/3}]`
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
- Resizing image to some fixed shape.
|
| 99 |
-
Representation: `[source_block , {"type": "resize_to", "size_x": 3, "size_y": 3}]`
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
- Swapping some colors.
|
| 103 |
-
Representation: `[source_block , {"type": "color_swap", "color_1": color_dict_1, "color_2": color_dict_2}]`
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
Etc.
|
| 107 |
-
|
| 108 |
-
- There is also one special type of blocks - `[{"type": "target", "k": I}]`. It is used when for the solving ot the task we need to use the block not presented on any of the input images but presented on all target images in the train examples.
|
| 109 |
-
|
| 110 |
-
### 2.1.3. Masks
|
| 111 |
-
|
| 112 |
-
Masks are binary images somehow derived from original images. Each mask is represented as a nested dict.
|
| 113 |
-
|
| 114 |
-
- Initial mask literally: `block == color`.
|
| 115 |
-
Representation: `{"operation": "none", "params": {"block": bloc_list,"color": color_dict}}`
|
| 116 |
-
bloc_list here is a list of transforms used to get the block for the mask generation
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
- Logical operations over different masks
|
| 120 |
-
Representation: `{"operation": "not", "params": mask_dict}, {"operation": "and", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}, {"operation": "or", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}, {"operation": "xor", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}`
|
| 121 |
-
|
| 122 |
-
- Mask with the original image's size, representing the smallest possible rectangle covering all pixels of a particular color.
|
| 123 |
-
Representation: `{"operation": "coverage", "params": {"color": color_dict}}`
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
- Mask with the original image's size, representing the largest or smallest connected block excluding the background.
|
| 127 |
-
Representation: {"operation": "max_block"}
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
You can find more information about existing abstractions and the code to generate them in `preprocessing.py`.
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
## 2.2 Predictors
|
| 134 |
-
|
| 135 |
-
I have created 32 different classes to solve different types of abstract task using the abstractions described earlier. All of them inherit from `Predictor` class.
|
| 136 |
-
|
| 137 |
-
The general logic of every predictor is described in the pseudo-code below (although, it can be different for some classes).
|
| 138 |
-
|
| 139 |
-
```
|
| 140 |
-
for n, (input_image, output_image) in enumerate(sample[‘train’]):
|
| 141 |
-
list_of_solutions = []
|
| 142 |
-
for possible_solution in all_possible_solutions:
|
| 143 |
-
if apply_solution(input_image, possible_solution) == output_image:
|
| 144 |
-
list_of_solutions.append(possible_solution)
|
| 145 |
-
if n == 0:
|
| 146 |
-
final_list_of_solutions = list_of_solutions
|
| 147 |
-
else:
|
| 148 |
-
final_list_of_solutions = intersection(list_of_solutions, final_list_of_solutions)
|
| 149 |
-
|
| 150 |
-
if len(final_list_of_solutions) == 0
|
| 151 |
-
return None
|
| 152 |
-
|
| 153 |
-
answers = []
|
| 154 |
-
for test_input_image in sample[‘test’]:
|
| 155 |
-
answers.append([])
|
| 156 |
-
for solution in final_list_of_solutions:
|
| 157 |
-
answers[-1].append(apply_solution(test_input_image, solution))
|
| 158 |
-
|
| 159 |
-
return answers
|
| 160 |
-
```
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
# 3. Kind of decorators or callbacks
|
| 166 |
-
|
| 167 |
-
I have also implemented some functionality, helping my predictors to solve a broader range of tasks. The way I applied them does not strictly qualify as decorators or callbacks but have similar ideas behind them.
|
| 168 |
-
|
| 169 |
-
## 3.1 Skip some train samples
|
| 170 |
-
If you pass to any predictor `{"skip_train": n}`, it will skip n input-output train pairs. It helps in a couple of cases:
|
| 171 |
-
- One of the train samples has a mistake in it.
|
| 172 |
-
- Train samples can be divided into 2+ categories, the predictor can solve the task only for one of them, and you are lucky, and the test is also from this group.
|
| 173 |
-
|
| 174 |
-
## 3.2 Rotate, Reflect, Roll
|
| 175 |
-
Sometimes it is useful to rotate, reflect, and/or roll the input and / or output image for your predictor to solve the case.
|
| 176 |
-
|
| 177 |
-
## 3.3 Eliminate Background
|
| 178 |
-
In some cases, there are parts of the image that contain relatively simple tasks that can be solved by existing predictors. `elim_background` option helps to work with these cases.
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
# 4. Optimization
|
| 183 |
-
|
| 184 |
-
Each time you add any additional abstraction, the number of possible variants, computational time and memory used growth very fast. To work with it, I used some tricks.
|
| 185 |
-
|
| 186 |
-
## 4.1 Caching
|
| 187 |
-
Usually, you can extract one block from the image with a lot of different ways. And there is no need to store all of them as well as there is no need to compute many times.
|
| 188 |
-
|
| 189 |
-
I store all the information about blocks in the following format:
|
| 190 |
-
|
| 191 |
-
```
|
| 192 |
-
{
|
| 193 |
-
"arrays”: {
|
| 194 |
-
array1_hash: {
|
| 195 |
-
"array”: numpy array representing the block
|
| 196 |
-
"params”: the list of all possible ways to generate this array
|
| 197 |
-
},
|
| 198 |
-
array2_hash: {
|
| 199 |
-
"array”: numpy array representing the block
|
| 200 |
-
"params”: the list of all possible ways to generate this array
|
| 201 |
-
},
|
| 202 |
-
…
|
| 203 |
-
"params”:{
|
| 204 |
-
params1_hash: corresponding_array_hash,
|
| 205 |
-
params2_hash: corresponding_array_hash,
|
| 206 |
-
….
|
| 207 |
-
}
|
| 208 |
-
}
|
| 209 |
-
```
|
| 210 |
-
|
| 211 |
-
Whenever I generate a new array, I compute its hash and check, whether I already calculated, if so, I just append the new params to the corresponding list, and add params hash to the `params` dict with the link to the array.
|
| 212 |
-
|
| 213 |
-
If I have a JSON-like object with parameters some abstract rule for block generation, first of all, I check whether the hash of this JSON is in "params" dict and if so, don't compute it but just take from the `arrays` dict.
|
| 214 |
-
|
| 215 |
-
The same logic is applied to masks.
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
## 4.2 Filtering
|
| 219 |
-
A high level of abstraction leads to situations when one transformation can be described in many different ways, and all of them will be equivalent and correct. It is ok, but it increases the computational complexity. That is why I do filtering.
|
| 220 |
-
The simplest example is about colors: let's imaging that all test and train images have four colors each, that means that `{"type": "min", "k": 0}` and `{"type": "max", "k": 3}` will always represent the same color for each image (but can represent different colors for different). This means that we can safely delete one of these options form all samples.
|
| 221 |
-
More generally, if two abstract representation of some object results in the same object for every image (train and test), we can ignore one of them.
|
| 222 |
-
This logic applies to all types of objects: colors, blocks, and masks.
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
## 4.3 Multiprocessing
|
| 226 |
-
Fast iterations are essential in this competition. That is why I have used multiprocessing wherever it is possible. I have used 24 threads locally for experiments and all 4 threads in kernels for faster submissions.
|
| 227 |
-
|
| 228 |
-
Another useful thing in multiprocessing – you can control the amount of RAM and computational time for every process and safely kill it if needed.
|
| 229 |
-
|
| 230 |
-
You can check the way I have done it in `submission_utils.py`.
|
| 231 |
-
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
# 5. Other useful ideas
|
| 235 |
-
|
| 236 |
-
## 5.1 Testing
|
| 237 |
-
This is the first Kaggle competition where I have covered almost everything I have done with unit tests.
|
| 238 |
-
|
| 239 |
-
I have often faced the situation when I improve core logic to solve some complex problems and later find out that some simple ones don't work anymore. After that, I have decided that I will add a unit test for most of the new tasks I have solved. Finally, I have 100+ tests in `test_predictots.py` that help me control that nothing is broken. I use the `pytest` library for it.
|
| 240 |
-
|
| 241 |
-
|
| 242 |
-
# 6. Final words
|
| 243 |
-
I have tried to describe everything I have done as clearly as possible, but I understand that the logic and code are not straightforward, so feel free to ask any questions.
|
| 244 |
-
|
| 245 |
-
I hope you have found any useful thoughts and ideas in my solution.
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
### Citation
|
| 249 |
-
|
| 250 |
-
```
|
| 251 |
-
@misc{larchenko2020abstract_reasoning,
|
| 252 |
-
author = {Ilia Larchenko},
|
| 253 |
-
title = {Abstraction and Reasoning Challenge Solution},
|
| 254 |
-
year = {2020},
|
| 255 |
-
publisher = {Zenodo},
|
| 256 |
-
doi = {10.5281/zenodo.14646751},
|
| 257 |
-
url = {https://doi.org/10.5281/zenodo.14646751},
|
| 258 |
-
note = {GitHub repository: \url{https://github.com/IliaLarchenko/abstract_reasoning}}
|
| 259 |
-
}
|
| 260 |
-
```
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: arc0-third-solution-image
|
| 3 |
+
emoji: 🧊
|
| 4 |
+
colorFrom: gray
|
| 5 |
+
colorTo: gray
|
| 6 |
+
sdk: docker
|
| 7 |
+
app_port: 7860
|
| 8 |
+
pinned: false
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
Auto-generated image-only Space used as a Docker registry by the
|
| 12 |
+
`Run Experiment (HF with Docker)` workflow. Do not edit by hand;
|
| 13 |
+
this repo is rewritten on every dispatch.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|