Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
eddf067
Add arrow from javacpp (also upgrades javacpp version)
agibsonccc Dec 29, 2019
c472256
Add basic table api
agibsonccc Dec 29, 2019
62cb047
Add op runner test
agibsonccc Dec 30, 2019
8749e1a
Fix byte pointer access
agibsonccc Dec 30, 2019
52ddfe6
Merge branch 'master' into ag_javacpparrow
agibsonccc Dec 30, 2019
2e9e595
Address comments
agibsonccc Dec 30, 2019
9158dfd
Merge branch 'ag_javacpparrow' of https://github.com/KonduitAI/deeple…
agibsonccc Dec 30, 2019
f03d2c5
Initial table api
agibsonccc Dec 30, 2019
faa004a
@raver119 can you review this? String changes and adds WIP column api
agibsonccc Dec 31, 2019
00f8fb8
Auto infer num words for utf8 and add test
agibsonccc Jan 1, 2020
eb23dc4
Add more utilities
agibsonccc Jan 1, 2020
4c60a54
Add tests for columns. Adds WIP row class.
agibsonccc Jan 2, 2020
64a0079
All tests pass. Still need to fill out table and row.
agibsonccc Jan 3, 2020
a3a5de4
Make all curren tests pass.
agibsonccc Jan 5, 2020
38ed19c
Add toNDArray and toList constructors/methods
agibsonccc Jan 5, 2020
5cdecd3
Merge branch 'master' of https://github.com/KonduitAI/deeplearning4j …
agibsonccc Jan 8, 2020
1538cae
Get rid of merge conflicts
agibsonccc Jan 8, 2020
40981a4
Merge branch 'master' into ag_javacpparrow
agibsonccc Jan 20, 2020
c75a5e1
Get rid of compilation errors
agibsonccc Jan 20, 2020
add666f
Merge branch 'ag_javacpparrow' of https://github.com/KonduitAI/deeple…
agibsonccc Jan 20, 2020
fd4c4ca
local
agibsonccc Feb 1, 2020
dacddb3
Merge branch 'master' of https://github.com/KonduitAI/deeplearning4j …
agibsonccc Feb 2, 2020
c9c29c2
Make tests run post update
agibsonccc Feb 2, 2020
cca6af4
Get rid of capacity/limit on arrow data
agibsonccc Feb 2, 2020
d1187e7
Merge branch 'master' into ag_javacpparrow
agibsonccc Feb 6, 2020
1b7de83
Merge branch 'master' into ag_javacpparrow
farizrahman4u Feb 11, 2020
337f784
Merge branch 'master' into ag_javacpparrow
farizrahman4u Feb 16, 2020
f3da385
Merge branch 'master' into ag_javacpparrow
saudet Apr 6, 2020
c7888f2
Merge branch 'master' into ag_javacpparrow
saudet Apr 6, 2020
f43b4af
Merge branch 'master' into ag_javacpparrow
raver119 Apr 7, 2020
ae9409f
- remove unwanted exception
raver119 Apr 7, 2020
f8fdbd2
few fixes here and there to make nd4j tests pass without crashes
raver119 Apr 7, 2020
b32a573
CUDA side of things updated
raver119 Apr 7, 2020
71acf5b
Minor fix for Nd4jArrowOpRunner
raver119 Apr 7, 2020
f3cf1a0
minor test fix
raver119 Apr 7, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -710,6 +710,17 @@ public Builder addColumn(ColumnMetaData metaData) {
return this;
}


/**
* Add a bytes column representing
* arbitrary bytes string data
* @param name the name of the column
* @return
*/
public Builder addColumnBytes(String name) {
return addColumn(new BinaryMetaData(name));
}

/**
* Add a String column with no restrictions on the allowable values.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ public static INDArray toArray(Collection<? extends Writable> record) {
INDArray arr = Nd4j.create(1, length);

int k = 0;
for (Writable w : record ) {
for (Writable w : record) {
if (w instanceof NDArrayWritable) {
INDArray toPut = ((NDArrayWritable) w).get();
arr.put(new INDArrayIndex[] {NDArrayIndex.point(0),
Expand Down
23 changes: 22 additions & 1 deletion datavec/datavec-arrow/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,29 @@
<packaging>jar</packaging>

<name>datavec-arrow</name>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-arrow</artifactId>
<version>${nd4j.version}</version>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>arrow-platform</artifactId>
<version>${arrow.javacpp.version}</version>
</dependency>
<dependency>
<groupId>org.datavec</groupId>
<artifactId>datavec-api</artifactId>
Expand Down
Loading