Filtering/sorting/paginating complicated data

Hello, I have in go data like:

bla := []map[string]interface{}{
    {
            Name:         "data1",
            Fields: map[string]interface{}{
                "Field1":        true,
                "Field2": map[string]interface{}{
                    "Key":           "foo",
                },
                "Field3": []map[string]interface{}{
                    {
                        "Key": "Bar",
                        "InnerField": []map[string]interface{}{
                            {
                                "Key": "Bar",
                                "Array": []int{1, 2}
                            },
                            {
                                "Key": "Baz",
                                "Array": []int{3, 4}
                            },
                        },
                    },
                    {
                        "Key": "Foo",
                        "InnerField": []map[string]interface{}{
                            {
                                "Key": "Rab",
                                "Array": []int{342, 234}
                            },
                            {
                                "Key": "Zab",
                                "Array": []int{534, 3453}
                            },
                        },
                    },
                },
           },
      },
}

It is only the example, there will be much more fields and the names of each field can be random (like Field3 or FieldNone). I have to keep it as map[string]interface I am not able to make from it struct :frowning:

My question is if I am able to in easy way to, for example, filtering,sorting,paginating over that complicated struct? In my head it should looks like:

filter := map[string]interface{}{
    "Fields.Fields3[].InnerField.Key": "Bar" 
}
res := db.Get(filter)

So I have got res like:

map[string]interface{}{
    Name:         "data1",
    Fields: map[string]interface{}{
        "Field1":        true,
        "Field2": map[string]interface{}{
            "Key":           "foo",
        },
        "Field3": []map[string]interface{}{
            {
                "Key": "Bar",
                "InnerField": []map[string]interface{}{
                    {
                        "Key": "Bar",
                        "Array": []int{1, 2}
                    },
                    {
                        "Key": "Baz",
                        "Array": []int{3, 4}
                    },
                },
            },
        },
    },
}

or

filter := map[string]interface{}{
    "Name": "data2" 
}
res := db.Get(filter)

res == []map[string]interface{} // res is empty

It looks like you want to perform operations like filtering, sorting, and pagination on the nested data structures you’ve provided. Since you can’t create a custom struct for this data, one way to do this is by using recursion and the reflect package.

Here’s a simple function that filters the data based on a filter map:

package main

import (
	"fmt"
	"reflect"
)

func filterData(data interface{}, filter map[string]interface{}, path []string) interface{} {
	if len(path) == 0 {
		if reflect.DeepEqual(data, filter) {
			return data
		}
		return nil
	}

	if m, ok := data.(map[string]interface{}); ok {
		if v, ok := m[path[0]]; ok {
			return filterData(v, filter, path[1:])
		}
	} else if arr, ok := data.([]map[string]interface{}); ok {
		result := []map[string]interface{}{}
		for _, e := range arr {
			if r := filterData(e, filter, path); r != nil {
				result = append(result, r.(map[string]interface{}))
			}
		}
		if len(result) > 0 {
			return result
		}
	}

	return nil
}

func main() {
	// Your 'bla' data

	filter := map[string]interface{}{
		"Key": "Bar",
	}

	filterPath := []string{"Fields", "Field3", "InnerField"}

	filtered := filterData(bla, filter, filterPath)

	fmt.Printf("%v\n", filtered)
}

This function traverses the data structure following the path given by filterPath. If it finds a match for the filter, it will return that element; otherwise, it will return nil.

This is just a starting point to help you with filtering. To implement sorting and pagination, you might need to extend this function or create new ones that work with your specific use case. Note that this implementation can be improved in terms of performance and code style, but it should provide a good starting point for handling the nested structures you’ve provided.