aidevices commited on
Commit
1264d09
·
verified ·
1 Parent(s): fba52a7

Sync third-solution for HF Jobs Docker image

Browse files
Files changed (1) hide show
  1. README.md +13 -260
README.md CHANGED
@@ -1,260 +1,13 @@
1
- [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.14646751.svg)](https://doi.org/10.5281/zenodo.14646751)
2
-
3
- This repository contains my solution of the [Abstraction and Reasoning Challenge](https://www.kaggle.com/c/abstraction-and-reasoning-challenge) on Kaggle, where my team took 3rd place. It is only my part of our final solution. Using it alone with blend with some public kernels, one could get a position somewhere in the higher silver. This approach also solves 138 and 96 samples from the train and valid sets, respectively. You can find the lists of solved cases in `solved.json`.
4
- `demo_notebook.ipynb` demonstrates how to make predictions using my code.
5
-
6
- You can also look into [Kaggle Notebook](https://www.kaggle.com/ilialar/5-tasks-part-of-3rd-place-solution) solving 5 test tasks with my solution.
7
-
8
- Even though the competition itself is very original, I hope the ideas I used can help someone in other applied tasks.
9
-
10
- # 0. Disclaimer
11
- This competition required a lot of tries and errors, and any time I choose between simple, high-quality production-level code and fast iteration, I decided latter. So the code itself is not optimal and unclear sometimes. But in this particular case, I think that ideas are more important than code itself.
12
-
13
- # 1. Introduction
14
- The task of the competition was "to create an AI that can solve reasoning tasks it has never seen before."
15
-
16
- This competition is highly specific and not similar to any other DS competitions I have participated in. This particular repository and the description below are addressed to those who participated in the competition or at least familiar with the task. I recommend you to read the information [on the official Kaggle page](https://www.kaggle.com/c/abstraction-and-reasoning-challenge), [github repository](https://github.com/fchollet/ARC) and / or [the original article about abstraction and reasoning](https://arxiv.org/abs/1911.01547)
17
-
18
-
19
- # 2. My approach
20
-
21
- In general, my method can be described as a domain-specific language (DSL) based. I have created the abstract representation of the colors, image blocks extracted from the original input image, and different binary masks. Using these abstractions, I wrote several "predictors" that try to derive a set of transformations used to get every output image from the corresponding input image and, if it succeeds – apply these transformations to test input to get the final answer.
22
-
23
- ## 2.1 Abstractions
24
- Each abstract representation of color, block, or mask is a JSON-like object that describes a set of transformations used to get this object from the original image. Below are several examples.
25
-
26
- ### 2.1.1 Colors
27
- I use a few ways to represent colors; below are some of them:
28
-
29
- - Absolute values. Each color is described as a number from 0 to 9.
30
- Representation: `{"type”: "abs”, "k”: 0}`
31
-
32
- - The numerical order of color in the list of all colors presented in the input image sorted (ascending or descending) by the number of pixels with these colors.
33
- Representation: `{"type”: "min”, "k”: 0}`, `{"type”: "max”, "k”: 0}`
34
-
35
- - The color of the grid if there is one on the input image.
36
- Representation: `{"type”: "grid_color”}`
37
-
38
- - The unique color in one of the image parts (top, bottom, left, or right part; corners, and so on).
39
- Representation: `{"type": "unique", "side": "right"}`, `{"type": "unique", "side": "tl"}`, `{"type": "unique", "side": "any"}`
40
-
41
- - No background color for cases where every input has only two colors and 0 is one of them for every image.
42
- Representation: `{"type": "non_zero"}`
43
-
44
- Etc.
45
-
46
-
47
- ### 2.1.2 Blocks
48
- Block is a 10-color image somehow derived from the input image. Each block is represented as a list of dicts; each dict describes some transformation of an image. One should apply all these transformations to the input image in the order they are presented in the list to get the block. Below are some examples.
49
-
50
- #### The first order blocks (generated directly from the original image):
51
-
52
- - The image itself.
53
- Representation: `[{"type": "original"}]`
54
-
55
- - One of the halves of the original image.
56
- Representation: `[{"type": "half", "side": "t"}]`, `[{"type": "half", "side": "b"}]`, `[{"type": "half", "side": "long1"}]`
57
-
58
- - The largest connected block excluding the background.
59
- Representation: `[{"type": "max_block", "full": 0}]`
60
-
61
- - The smallest possible rectangle that covers all pixels of a particular color.
62
- Representation: `[{"type": "color_max", "color": color_dict}]`
63
- color_dict – means here can be any abstract representation of color, described in 2.1.1.
64
-
65
-
66
- - Grid cell.
67
- Representation: `[{"type": "grid", "grid_size": [4,4],"cell": [3, 1],"frame": True}]`
68
-
69
-
70
- - The pixel with particular coordinates.
71
- Representation: `[{"type": "pixel", "i": 1, "j": 4}]`
72
-
73
-
74
- Etc.
75
-
76
-
77
- #### The second-order blocks – generated by applying some additional transformations to the other blocks:
78
-
79
-
80
- - Rotation.
81
- Representation: `[source_block ,{"type": "rotation", "k": 2}]`
82
- `source_block` _means that there can be one or several dictionaries, used to generate some source block from the original input image, then the rotation is applied to this source block_
83
-
84
-
85
- - Transposing.
86
- Representation: `[source_block ,{"type": "transpose"}]`
87
-
88
-
89
- - Edge cutting.
90
- Representation: `[source_block ,{"type": "cut_edge", "l": 0, "r": 1, "t": 1, "b": 0}]`
91
- _In this example, we cut off 1 pixel from the left and one pixel from the top of the image._
92
-
93
-
94
- - Resizing image with some scale factor.
95
- Representation: `[source_block , {"type": "resize", "scale": 2}]`, `[source_block , {"type": "resize", "scale": 1/3}]`
96
-
97
-
98
- - Resizing image to some fixed shape.
99
- Representation: `[source_block , {"type": "resize_to", "size_x": 3, "size_y": 3}]`
100
-
101
-
102
- - Swapping some colors.
103
- Representation: `[source_block , {"type": "color_swap", "color_1": color_dict_1, "color_2": color_dict_2}]`
104
-
105
-
106
- Etc.
107
-
108
- - There is also one special type of blocks - `[{"type": "target", "k": I}]`. It is used when for the solving ot the task we need to use the block not presented on any of the input images but presented on all target images in the train examples.
109
-
110
- ### 2.1.3. Masks
111
-
112
- Masks are binary images somehow derived from original images. Each mask is represented as a nested dict.
113
-
114
- - Initial mask literally: `block == color`.
115
- Representation: `{"operation": "none", "params": {"block": bloc_list,"color": color_dict}}`
116
- bloc_list here is a list of transforms used to get the block for the mask generation
117
-
118
-
119
- - Logical operations over different masks
120
- Representation: `{"operation": "not", "params": mask_dict}, {"operation": "and", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}, {"operation": "or", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}, {"operation": "xor", "params": {"mask1": mask_dict 1, "mask2": mask_dict 2}}`
121
-
122
- - Mask with the original image's size, representing the smallest possible rectangle covering all pixels of a particular color.
123
- Representation: `{"operation": "coverage", "params": {"color": color_dict}}`
124
-
125
-
126
- - Mask with the original image's size, representing the largest or smallest connected block excluding the background.
127
- Representation: {"operation": "max_block"}
128
-
129
-
130
- You can find more information about existing abstractions and the code to generate them in `preprocessing.py`.
131
-
132
-
133
- ## 2.2 Predictors
134
-
135
- I have created 32 different classes to solve different types of abstract task using the abstractions described earlier. All of them inherit from `Predictor` class.
136
-
137
- The general logic of every predictor is described in the pseudo-code below (although, it can be different for some classes).
138
-
139
- ```
140
- for n, (input_image, output_image) in enumerate(sample[‘train’]):
141
- list_of_solutions = []
142
- for possible_solution in all_possible_solutions:
143
- if apply_solution(input_image, possible_solution) == output_image:
144
- list_of_solutions.append(possible_solution)
145
- if n == 0:
146
- final_list_of_solutions = list_of_solutions
147
- else:
148
- final_list_of_solutions = intersection(list_of_solutions, final_list_of_solutions)
149
-
150
- if len(final_list_of_solutions) == 0
151
- return None
152
-
153
- answers = []
154
- for test_input_image in sample[‘test’]:
155
- answers.append([])
156
- for solution in final_list_of_solutions:
157
- answers[-1].append(apply_solution(test_input_image, solution))
158
-
159
- return answers
160
- ```
161
-
162
-
163
-
164
-
165
- # 3. Kind of decorators or callbacks
166
-
167
- I have also implemented some functionality, helping my predictors to solve a broader range of tasks. The way I applied them does not strictly qualify as decorators or callbacks but have similar ideas behind them.
168
-
169
- ## 3.1 Skip some train samples
170
- If you pass to any predictor `{"skip_train": n}`, it will skip n input-output train pairs. It helps in a couple of cases:
171
- - One of the train samples has a mistake in it.
172
- - Train samples can be divided into 2+ categories, the predictor can solve the task only for one of them, and you are lucky, and the test is also from this group.
173
-
174
- ## 3.2 Rotate, Reflect, Roll
175
- Sometimes it is useful to rotate, reflect, and/or roll the input and / or output image for your predictor to solve the case.
176
-
177
- ## 3.3 Eliminate Background
178
- In some cases, there are parts of the image that contain relatively simple tasks that can be solved by existing predictors. `elim_background` option helps to work with these cases.
179
-
180
-
181
-
182
- # 4. Optimization
183
-
184
- Each time you add any additional abstraction, the number of possible variants, computational time and memory used growth very fast. To work with it, I used some tricks.
185
-
186
- ## 4.1 Caching
187
- Usually, you can extract one block from the image with a lot of different ways. And there is no need to store all of them as well as there is no need to compute many times.
188
-
189
- I store all the information about blocks in the following format:
190
-
191
- ```
192
- {
193
- "arrays”: {
194
- array1_hash: {
195
- "array”: numpy array representing the block
196
- "params”: the list of all possible ways to generate this array
197
- },
198
- array2_hash: {
199
- "array”: numpy array representing the block
200
- "params”: the list of all possible ways to generate this array
201
- },
202
- …
203
- "params”:{
204
- params1_hash: corresponding_array_hash,
205
- params2_hash: corresponding_array_hash,
206
- ….
207
- }
208
- }
209
- ```
210
-
211
- Whenever I generate a new array, I compute its hash and check, whether I already calculated, if so, I just append the new params to the corresponding list, and add params hash to the `params` dict with the link to the array.
212
-
213
- If I have a JSON-like object with parameters some abstract rule for block generation, first of all, I check whether the hash of this JSON is in "params" dict and if so, don't compute it but just take from the `arrays` dict.
214
-
215
- The same logic is applied to masks.
216
-
217
-
218
- ## 4.2 Filtering
219
- A high level of abstraction leads to situations when one transformation can be described in many different ways, and all of them will be equivalent and correct. It is ok, but it increases the computational complexity. That is why I do filtering.
220
- The simplest example is about colors: let's imaging that all test and train images have four colors each, that means that `{"type": "min", "k": 0}` and `{"type": "max", "k": 3}` will always represent the same color for each image (but can represent different colors for different). This means that we can safely delete one of these options form all samples.
221
- More generally, if two abstract representation of some object results in the same object for every image (train and test), we can ignore one of them.
222
- This logic applies to all types of objects: colors, blocks, and masks.
223
-
224
-
225
- ## 4.3 Multiprocessing
226
- Fast iterations are essential in this competition. That is why I have used multiprocessing wherever it is possible. I have used 24 threads locally for experiments and all 4 threads in kernels for faster submissions.
227
-
228
- Another useful thing in multiprocessing – you can control the amount of RAM and computational time for every process and safely kill it if needed.
229
-
230
- You can check the way I have done it in `submission_utils.py`.
231
-
232
-
233
-
234
- # 5. Other useful ideas
235
-
236
- ## 5.1 Testing
237
- This is the first Kaggle competition where I have covered almost everything I have done with unit tests.
238
-
239
- I have often faced the situation when I improve core logic to solve some complex problems and later find out that some simple ones don't work anymore. After that, I have decided that I will add a unit test for most of the new tasks I have solved. Finally, I have 100+ tests in `test_predictots.py` that help me control that nothing is broken. I use the `pytest` library for it.
240
-
241
-
242
- # 6. Final words
243
- I have tried to describe everything I have done as clearly as possible, but I understand that the logic and code are not straightforward, so feel free to ask any questions.
244
-
245
- I hope you have found any useful thoughts and ideas in my solution.
246
-
247
-
248
- ### Citation
249
-
250
- ```
251
- @misc{larchenko2020abstract_reasoning,
252
- author = {Ilia Larchenko},
253
- title = {Abstraction and Reasoning Challenge Solution},
254
- year = {2020},
255
- publisher = {Zenodo},
256
- doi = {10.5281/zenodo.14646751},
257
- url = {https://doi.org/10.5281/zenodo.14646751},
258
- note = {GitHub repository: \url{https://github.com/IliaLarchenko/abstract_reasoning}}
259
- }
260
- ```
 
1
+ ---
2
+ title: arc0-third-solution-image
3
+ emoji: 🧊
4
+ colorFrom: gray
5
+ colorTo: gray
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ ---
10
+
11
+ Auto-generated image-only Space used as a Docker registry by the
12
+ `Run Experiment (HF with Docker)` workflow. Do not edit by hand;
13
+ this repo is rewritten on every dispatch.