README.Rmd

---
title: "IP33 results conversion"
output: github_document
---

Converting results of IP33 to spatial data for inclusion in data viewer.

This script reads data from three text (csv) files: 

- [Copepod data_2018to20.csv](#copepod)
- [Fish acoustic abd data 2018to20_copy4OBAMA.csv](#acoustic)
- [trawl_fish abd_size quantileranges_copy_OBAMAIP33.csv](#trawl)

The data from each file is converted to spatial data, using the coordinates included. The spatial data are then [exported](#export) as shape files.

The results are gathered in this [zip file](obama_next_ip33.zip).

For a simple R script without any markdown, use [ip33_convert_vector.R](R/ip33_convert_vector.R).

```{r libraries, message=FALSE, warning=FALSE}
library(dplyr)
library(tidyr)
library(sf)
library(ggplot2)
library(patchwork)
library(zip)
```

## Copepod data_2018to20.csv []{#copepod}

Read the data file.

```{r read-data-1}
file1 <- "Copepod data_2018to20.csv"

df1 <- read.csv(paste0("data/",file1), sep=",", header=T)

print(df1 %>% slice(1:10), row.names = FALSE)
```

Create spatial data.

```{r convert-sf-1}
shp1 <- sf::st_as_sf(df1, coords = c("lon","lat"), crs=4326)
```

Plot the data.

```{r plot-data-1}
p1 <- ggplot() +
  geom_sf(data = shp1, aes(colour=cop_abun_m3), size=0.2) +
  scale_x_continuous(breaks=c(-6,-4,-2,0)) +
  scale_y_continuous(breaks=c(49,50,51,52)) +
  theme_minimal()  +
  scale_color_distiller(palette = "Spectral")

ggsave(p1, filename="plot/Copepod data_2018to20.png", 
       height=12, width=16, units="cm", dpi=300)
```

![](plot/Copepod data_2018to20.png){width=500}

## Fish acoustic abd data 2018to20_copy4OBAMA.csv []{#acoustic}

Read the data file.

```{r read-data-2}
file2 <- "Fish acoustic abd data 2018to20_copy4OBAMA.csv"
df2 <- read.csv(paste0("data/",file2), sep=",", header=T)
print(df2 %>% slice(1:10), row.names = FALSE)
```

Create spatial data.

```{r convert-sf-2}
shp2 <- sf::st_as_sf(df2, coords = c("longiture","latitud"), crs=4326)
```

Plot the data.

```{r plot-data-2}
p2 <- ggplot() +
  geom_sf(data = shp2, aes(colour=abundance_indv_m2), size=0.2) +
  theme_minimal()  +
  scale_x_continuous(breaks=c(-6,-4,-2,0)) +
  scale_y_continuous(breaks=c(49,50,51,52)) +
  facet_wrap(taxa~., ncol=3) +
  theme(legend.position.inside = c(0.85, 0.2),
        legend.position = "inside")  +
  scale_color_distiller(palette = "Spectral")

ggsave(p2, filename="plot/Fish acoustic abd data 2018to20_copy4OBAMA.png", 
       height=12, width=16, units="cm", dpi=300)
```

![](plot/Fish acoustic abd data 2018to20_copy4OBAMA.png){width=500}

## trawl_fish abd_size quantileranges_copy_OBAMAIP33.csv []{#trawl}

Read the data file.

```{r read-data-3}
file3 <- "trawl_fish abd_size quantileranges_copy_OBAMAIP33.csv"
df3 <- read.csv(paste0("data/",file3), sep=",", header=T)
print(df3 %>% slice(1:10), row.names = FALSE)
```

Create spatial data.

```{r convert-sf-3}
shp3 <- sf::st_as_sf(df3, coords = c("lon","lat"), crs=4326)
```

Plot the data.

```{r plot-data-3}
p3a <- ggplot() +
  geom_sf(data = shp3, aes(colour=pct_below_Q1), size=1) +
  theme_minimal() +
  scale_x_continuous(breaks=c(-6,-4,-2,0)) +
  scale_y_continuous(breaks=c(49,50,51,52)) +
  facet_wrap(fldMainSpeciesCode~., ncol=2) +
  theme(legend.position = "bottom", 
        legend.title = element_text(size=rel(0.8))) +
  scale_color_distiller(palette = "Spectral", direction = 1)

p3b <- ggplot() +
  geom_sf(data = shp3, aes(colour=pct_above_Q3), size=1) +
  theme_minimal() +
  scale_x_continuous(breaks=c(-6,-4,-2,0)) +
  scale_y_continuous(breaks=c(49,50,51,52)) +
  facet_wrap(fldMainSpeciesCode~., ncol=2) +
  theme(legend.position = "bottom", 
        legend.title = element_text(size=rel(0.8))) +
  scale_color_distiller(palette = "Spectral")

p3 <- p3a + p3b

ggsave(p3, filename="plot/trawl_fish abd_size quantileranges_copy_OBAMAIP33.png",
       height=16, width=16, units="cm", dpi=300)
```

![](plot/trawl_fish abd_size quantileranges_copy_OBAMAIP33.png){width=500}

## Save shape files []{#export}

Some of the column names in the data frames need to be shortened before exporting to shape file.

We will keep a record of the changes and store them in a text file.

```{r save-shape-files}
# > names(shp1)
# [1] "X" "bindatetime"  "cop_abun_m3" "copepod_length_geomean_mm"     
# [5] "Lifeform_large_copepod.....2mm" "lifeform_small_copepod....2mm"  "geometry"        
names_dat1 <- names(shp1)
names(shp1) <- c("id","datetime", "abund_m3", "len_mm", "n_large", "n_small", "geometry") 
names_shp1 <- names(shp1)

# > names(shp2)
# [1] "date" "time" "taxa" "instrument" "abundance_indv_m2" "X"                
# [7] "X.1" "X.2" "X.3" "geometry"         
names_dat2 <- names(shp2)
names(shp2)[names(shp2)=="abundance_indv_m2"] <- "abund_m2"
names_shp2 <- names(shp2)

# > names(shp3)
# [1] "X" "fldCruiseName" "fldCruiseStationNumber" "fldMainSpeciesCode"  "year"                  
# [6] "fldDateTimeShot" "total_fish" "pct_below_Q1" "pct_above_Q3"  "geometry"              
names_dat3 <- names(shp3)
names(shp3) <- c("id","cruisename", "stn_no", "spcs_code", "year",
                 "time", "tot_fish", "pct_lt_Q1","pct_gt_Q3", "geometry")
names_shp3 <- names(shp3)

file1 <- rep("Copepod data_2018to20.shp", length(names_shp1))
file2 <- rep("Fish acoustic abd data 2018to20_copy4OBAMA.shp", length(names_shp2))
file3 <- rep("trawl_fish abd_size quantileranges_copy_OBAMAIP33.shp", length(names_shp3))

df <- data.frame(file = c(file1, file2, file3), 
                 column_shape=c(names_shp1,names_shp2,names_shp3),
                 column_data=c(names_dat1,names_dat2,names_dat3))

# save the table with column names
write.table(df, file = "shp/columns_renamed.csv", row.names = F, sep=",")

# save the shape files
sf::st_write(shp1, "shp/Copepod data_2018to20.shp", append=F)
sf::st_write(shp2, "shp/Fish acoustic abd data 2018to20_copy4OBAMA.shp", append=F)
sf::st_write(shp3, "shp/trawl_fish abd_size quantileranges_copy_OBAMAIP33.shp", append=F)
```

The column names conversions are in [columns_renamed.csv](shp/columns_renamed.csv)

This file has 3 columns:

* _file_ - the name of the shape file 
* _column_shape_ - the name of a column in the shape file
* _column_data_ - the name of the corresponding column in the original data file*

\*Note: column names beginning with **X** in _column_data_ were automatically 
assigned by the ´read.csv()´ function. This is because these columns did not 
have names in the original csv files.

## Zip the files

Collect shape files and png files in a zip file.

```{r zip-files}

files_plot <- list.files(path="plot", full.names = T) 
files_shp <- list.files(path="shp", full.names = T) 

files_zip <- c(files_shp, files_plot)

zip(zipfile="obama_next_ip33.zip", files=files_zip)

```