Add support for NamedArray

[s-celles]

Hello,

I'm using FreqTables.jl freqtable.
This function outputs NamedArray objects.
Maybe it could be a good idea to add support for NamedArrays into IterableTables.

Kind regards

[davidanthoff]

Do these have a table structure, though? I'm just not sure...

[s-celles]

If dimension is no more than 2, I think so.
If dimension is greater, than an error should be raised.
Your comment is very interesting as it shows that there is a room for a project (comparable to yours) that could deal with converting n dimensional datastructures such as NamedArray or IndexedTables currently

[davidanthoff]

Interesting, interesting... I think the general pattern he might be some way to get rows out of a matrix, both a named and a normal matrix. For Query.jl integration it would be nice if these rows were tuples and named tuples respectively. But I'm not sure that is in general a good idea, I'm not sure whether very large tuples (if there are very many rows) work well...

It would have to be a special function, in any case, something like rows. I'll have to think a bit more about that scenario.

[s-celles]

See davidavdav/NamedArrays.jl#55 about implementing an Iterator of NamedTuples for NamedArrays

[s-celles]

Pinging @davidavdav

[davidavdav]

You rang?

I am a bit out of context, I've tried to read up to the references, but it is not clear to me what an iterator of named tuples would do with a NamedArray. Do you want to be able to iterate over a dimension on a NamedArray, thereby getting a tuple of the name and the array slice of one dimension lower, and do you want this to be of type Iterator for NamedTuples?

[s-celles]

I think we should first tackle NamedArrays with dimension of 2 and see how such a NamedArray can be converted to a DataFrame (or any table-like data structure that IterableTables deals with as a sink)
IterableTable can easily use a source which consist of any iterator who produces elements of type NamedTuple.

The problem of dimensions greater than 2 can't be handle in this project (I think... or at least not for now).

[nalimilan]

In R tables of arbitrary dimensionality can be converted to data frames. Each dimension gets transformed into a column, and an additional columns holds the values of the array entries.

[davidanthoff]

@nalimilan so all the columns except the last one would hold the indices of that respective dimension? For example, this matrix:

1  2  3
4  5  6
7  8  9

Would be transformed into this table:

Dim1	Dim2	Value
1	1	1
1	2	2
1	3	3
2	1	4
2	2	5
2	3	6
3	1	7
3	2	8
3	3	9

? I think that might be a really good general solution. It would still be nice if there was some easy way to handle a matrix differently, i.e. keep the table structure of the matrix, but one that could be done by say a row function.

I guess this also somehow interacts with this idea of how associative are handled. This would essentially treat an array as an associative, with the dimension indices as the key. I didn't follow that debate in detail, though...

[nalimilan]

Yes, that's it, though it's even clearer when there are dimension names rather than indices.

[davidavdav]

Ik looks like an R-style NamedArray -> DataFrame export would generally be beneficial.

[s-celles]

If we can have NamedArray export to IndexedTables (and vice versa) it will be great.
Adding support of NamedArray to IterableTables will help to achieve this goal.