Skip to content
Open

L6 #271

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d270212
Added test file, added PyCharm files to gitignore
Oct 6, 2020
d6774cd
Added tasks 1-3
Nov 19, 2020
fefb3f6
Added example db connection and svg saving
Nov 20, 2020
57b24ad
Added task 4 and helper function in api file
Nov 20, 2020
1683e58
Added task 5, code cleanup
Nov 20, 2020
1f6ec05
Updated generated charts to look better
Nov 20, 2020
20ff52c
Deleted obsolete files
Nov 20, 2020
e394793
Added space at the end of gitignore
Nov 20, 2020
bb3c1e3
Added sample flask app
Dec 20, 2020
16dbf67
Updated db_connection to include method which will update missing rat…
Dec 20, 2020
54e5900
Added task 1 and 2 with exception handling
Dec 20, 2020
c019c8a
Added task 4
Dec 20, 2020
a8d465f
Update rates from api at given time every day, cache rates
Dec 21, 2020
27ebf46
Add task 3
Dec 21, 2020
204ccbb
Add readme file
Dec 21, 2020
423de69
Update README.md
GMyjak Dec 21, 2020
1b23e77
Updated headers in readme.md
GMyjak Dec 21, 2020
cc43718
Added spaces in models in readme.md
GMyjak Dec 21, 2020
5de5e28
Code cleanup, added screenshots
Dec 21, 2020
2d3a54a
Linked all posted screenshots to readme, changed directory name
Dec 21, 2020
9cc0e62
Removed obsolete files
Dec 21, 2020
67aa22f
Restructure project, add sample React app
Jan 14, 2021
38e345f
Added routing, pages scaffolds and navbar
Jan 21, 2021
52d9545
Added task 1 api description, fixed typo in README
Jan 21, 2021
a21b4c2
Scaffold Rates page
Jan 22, 2021
27e1fdd
Add API connection, data display on Rates page and fill Income page
Jan 23, 2021
ee493b6
Update readme, fixed some typos
Jan 23, 2021
f87646b
Minor cleanup
Jan 25, 2021
bc9275b
Hide graph with less than 2 data entries
Jan 25, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,144 @@
.DS_Store
.idea
.vscode
Lista2/
Lista3/

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/
119 changes: 119 additions & 0 deletions Lista5/db_connection.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
import pyodbc as db
import datetime
from api_connection import get_average_currency_rates_between


# Database created with:
# https://www.sqlservertutorial.net/sql-server-sample-database/
conn = db.connect('Driver={SQL Server};'
'Server=DESKTOP-AKTNFDK;'
'Database=BikeStores;'
'Trusted_Connection=yes;')


# Zadanie 4 - modyfikacja istniejcej bazy danych
def create_rates_table(data):
cursor = conn.cursor()
cursor.execute('CREATE TABLE BikeStores.dbo.rates ('
'measure_date DATE NOT NULL PRIMARY KEY,'
'measure_rate DECIMAL (7, 4) NOT NULL,'
'interpolated BIT NOT NULL);')
conn.commit()

update_rates_table(data)

# Added option to add interpolation from db in case update_rates_table_to_today is called on weekend
def update_rates_table(data, interpolate_date=None):
cursor = conn.cursor()
query = 'INSERT INTO BikeStores.dbo.rates(measure_date,measure_rate, interpolated) VALUES(?,?,?)'

for idx in range(len(data)):
cursor.execute(query, (data[idx]['effectiveDate'], float(data[idx]['mid']), 0))
if idx < len(data) - 1:
date_idx0 = datetime.datetime.strptime(data[idx]['effectiveDate'], '%Y-%m-%d')
date_idx1 = datetime.datetime.strptime(data[idx+1]['effectiveDate'], '%Y-%m-%d')
date_diff = (date_idx1 - date_idx0).days
for excess in range(date_diff - 1):
date_idx0 += datetime.timedelta(days=1)
cursor.execute(query, (date_idx0, float(data[idx]['mid']), 1))

# interpolate_date -> date of last rate that will be stored in db
if interpolate_date != None:
# if there is option, interpolate from data list passed as arg
if (len(data) > 0):
last_date = datetime.datetime.strptime(data[-1]['effectiveDate'], '%Y-%m-%d')
for excess in range((interpolate_date - last_date).days):
last_date += datetime.timedelta(days=1)
cursor.execute(query, (last_date, float(data[-1]['mid']), 1))
# otherwise take last rate from db
else:
cursor.execute('SELECT TOP (1) [measure_date],[measure_rate] FROM [BikeStores].[dbo].[rates] ORDER BY [measure_date] DESC;')
last_tuple = cursor.fetchone()
if last_tuple != None:
last_date = datetime.datetime.strptime(last_tuple[0], '%Y-%m-%d')
for excess in range((interpolate_date - last_date).days):
last_date += datetime.timedelta(days=1)
cursor.execute(query, (last_date, float(last_tuple[1]), 1))


conn.commit()


def get_profit_in_currencies(date_from, date_to):
cursor = conn.cursor()
cursor.execute("""SELECT [orders].[order_date] AS day_in_year,
SUM([order_items].[list_price]*[order_items].[quantity]) AS profit_usd,
SUM([order_items].[list_price]*[order_items].[quantity]*[rates].[measure_rate]) AS profit_pln
FROM [BikeStores].[sales].[orders]
JOIN [BikeStores].[sales].[order_items]
ON [BikeStores].[sales].[orders].[order_id] = [BikeStores].[sales].[order_items].[order_id]
JOIN [BikeStores].[dbo].[rates]
ON [BikeStores].[sales].[orders].[order_date] = [BikeStores].[dbo].[rates].[measure_date]
GROUP BY [orders].[order_date]
HAVING [orders].[order_date] BETWEEN '""" + date_from + "' AND '" + date_to +
'\'ORDER BY [orders].[order_date]')

result = []
for row in cursor:
result.append({'date':row[0], 'profit_usd':row[1], 'profit_pln':int(row[2]*100)/100})

return result


def update_rates_table_to_day(day):
cursor = conn.cursor()
# Get most recent date in table
cursor.execute("""SELECT TOP (1) [measure_date]
FROM [BikeStores].[dbo].[rates]
ORDER BY [measure_date] DESC""")

last_date = cursor.fetchone()
if last_date != None:
last_date = datetime.datetime.strptime(last_date[0], '%Y-%m-%d')
if (day > last_date):
table_data = get_average_currency_rates_between('USD', last_date + datetime.timedelta(days=1), day)
update_rates_table(table_data, day)


def get_rates_from_to(date_from, date_to):
cursor = conn.cursor()
cursor.execute('SELECT * FROM [BikeStores].[dbo].[rates] '
'WHERE [measure_date] >= \'' + date_from + '\'' +
' AND [measure_date] <= \'' + date_to + '\';')
result = []
for row in cursor:
result.append({'date':datetime.datetime.strptime(row[0], '%Y-%m-%d'),'rate':row[1],'interpolated':False if row[2] == 0 else True})
return result


def initialize_db():
try:
table_data = get_average_currency_rates_between('USD', datetime.date(2015, 12, 20), datetime.date(2018, 1, 4))
create_rates_table(table_data)
except Exception as ex:
print("Error while initializing db")


if __name__ == '__main__':
# Add database table and fill it
initialize_db()
72 changes: 72 additions & 0 deletions currency-api-project/back-end/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Języki skryptowe lista 5

## Ogólne informacje
API utworzone według wymagań listy 5

## Instalacja paczek

### Komendy do wywołania:
*pip install flask
pip install flask-limiter
pip install pyodbc
pip install requests*

### Na Windowsie najnowsza wersja numpy powodowała problemy
*pip uninstall numpy
pip install numpy==1.19.3*

## Uruchomienie
Po zainstalowaniu paczek:
Skonfigurować połączenie z bazą danych w pliku db_connection.py. Użyta baza: https://www.sqlservertutorial.net/sql-server-sample-database/
W przypadku pierwszego uruchomienia, wywołać skrypt *db_connection.py*. Dodaje on tabelę rates i uzupełnia ją danymi startowymi.
Windows - *python app.py*
Linux, Mac - *python3 app.py*
Serwer będzie dostępny pod adresem http://localhost:5000

## Endpointy

### Zadanie 1 i 2
### GET /rates/from/<date_from>/to/<date_to>
Gdzie <date_from> i <date_to> to daty w formacie RRRR-MM-DD
Status 200: Zwraca listę rate_dto
Status 400: Dla niepoprawnie sformatowanych dat
Status 400: Jeśli pojawią się dni spoza zakresu

Model rate_dto:
date : datetime RRRR-MM-DD --> Data dla której zwracany jest kurs
rate : float --> Kurs PLN-USD
interpolated : bool --> Czy wartość została przeniesiona z poprzedniego dnia z powodu braku danych

### Zadanie 3
### GET /profits/day/<date_from>/to/<date_to>
Gdzie <date_from> i <date_to> to daty w formacie RRRR-MM-DD
Status 200: Zwraca listę profit_dto
Status 400: Dla niepoprawnie sformatowanych dat
Status 400: Jeśli pojawią się dni spoza zakresu
Dla dni, w których nie odnotowano żadnego zysku, nic nie zwraca

Model profit_dto:
date : datetime RRRR-MM-DD --> Data dla której zwracane są wyniki
profit_usd : float --> Dochód z danego dnia w USD
profit_pln : float --> Dochód z danego dnia w PLN

## Informacje techniczne
* Żeby serwer nie został zapchany, stosowane jest ograniczenie 12 zapytań na minutę dla każdego użytkownika. W przypadku przekroczenia tej wartości, zwracany jest status code 429.
* Zakres dat obsługiwany przez api: od 2015-12-21 do dzień wcześniej względem dnia poprzedniego
* Ze względu na małą ilość danych oraz ich niezmienność, wszystkie zwracane dane są przechowywane w cache. Cache jest odświeżane codziennie o 21 (app.py -> update_at_hour)
* Najnowsze kursy są pobierane codziennie o 21 (app.py -> update_at_hour). Taka godzina gwarantuje, że NBP uzupełni swoje dane na ten dzień i mechanizm odświeżający bazę danych nie wstawi wartości interpolated dla danego dnia

## Screeny

### Poprawna odpowiedź od /rates -> 200
![Alt text](req1.png)
### Data wykracza spoza dozwolonego zakresu -> 400
![Alt text](req2.png)
### Data początkowa jest większa niż data końcowa -> 400
![Alt text](req3.png)
### Zabezpieczenie antyspamowe -> 429
![Alt text](req4.png)
### Niepoprawne daty ('XD', 'ASDF') -> 400
![Alt text](req5.png)
### Poprawna odpowiedź od /profits -> 200
![Alt text](req6.png)
17 changes: 17 additions & 0 deletions currency-api-project/back-end/api_connection.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import requests
import datetime


def get_average_currency_rates_between(currency, first_day, last_day):
url = 'http://api.nbp.pl/api/exchangerates/rates/A/' + currency + '/'
result = []
end_date = last_day
days = (last_day - first_day).days
while days > 0:
begin_date = end_date - datetime.timedelta(days=min(days, 92))
response = requests.get(url + begin_date.strftime('%Y-%m-%d') + '/' + end_date.strftime('%Y-%m-%d'))
if response.status_code == 200:
result = response.json()['rates'] + result
days -= 93
end_date = end_date - datetime.timedelta(days=93)
return result
Loading