Encounter a problem while running the ACORN example.

[AdeelAslamUnimore]

Hi,
I encounter a problem while running the ACORN example.

Example Code Missing metadata Parameter in IndexACORNFlat Constructor

Description:
The provided example for IndexACORNFlat is missing the required std::vector<int>& metadata parameter.

The example should initialize the IndexACORNFLAT:

std::vector<int> metadata.............; 
faiss::IndexACORNFlat acorn_gamma(d, M, gamma,  metadata, M_beta);

instead of.
faiss::IndexACORNFlat acorn_gamma(d, M, gamma, M_beta);

Error encountered:
Without metadata, the example does not compile.

Request:
Would it be possible to update the complete example using random values instead of the fvecs file? This would greatly help in making the example more understandable. Thank you for being so considerate!

[gongyguo]

I have similar questions, could you provide a simple and complete example for testing? Also, I noticed that the metadata is constructed by using 1D vector, can you provide some instructions on how to deal with higher dimensional Structured Data, such as datasets like LAION and Tripclick? Thanks for your help!

[hailangx]

It seems the metadata(or lables ) are unified into 1d, all the labels are preprocessed and feed into the search .

seems @jamjpan is also testing it, are you abel to confirm ?

[jamjpan]

I agree with @haiyangxu @AdeelAslamUnimore @gongyguo

I am running ACORN with:

faiss::IndexACORNFlat  acorn(count_vectors, M, gamma, flattened_attributes, M_beta);
acorn.add(count_vectors, flattened_vectors);
acorn.search(count_queries, flattened_queries, k, topkdis, topkids, bitmap);

where flattened_attributes is a 1-by-n std::vector<int> for n embedding vectors that associates one integer label per vector. But I don't know how this structure is actually used: If I pass a std::vector of all zeros, I seem to get the same search results as passing the real integer labels. But if I pass an empty std::vector, I get a segfault.

On the other hand, the search method seems to be relying on bitmap which for me is an m-by-n char*array for m queries and n embedding vectors. Each value of the array is either 0 or 1 indicating whether the corresponding embedding vector satisfies the query predicate or not. This seems like it is doing the real work and seems like it must be done as a pre-processing step before search is called.

[hailangx]

@jamjpan That's was what I am read from the code, the bitmap(filter_ids_map) is a preprocessing of the query labels to base points' label.
then if you preprocessing the data, any kind of filters can be done with this preprocessing , the code just give a simple equal check, but it can be complex to check if query label: aq[xq] matches base points' label: metadata[xb]

std::vector<char> filter_ids_map(nq * N);
for (int xq = 0; xq < nq; xq++) {
    for (int xb = 0; xb < N; xb++) {
        filter_ids_map[xq * N + xb] = (bool) (metadata[xb] == aq[xq]);
    }
}```