Skip to content

TODO.md

Latest commit

 

History

History
77 lines (67 loc) · 3.13 KB

TODO.md

File metadata and controls

77 lines (67 loc) · 3.13 KB
Raw

TODO list:

  • Profile via prof on linux or gprof in mingw...

  • New optimised dynamic table inserts

  • Redo dynamic table inserts

  • In debug SIGTRAP on flushing, fix

  • Names for scheduled tasks

  • Save scheduler table (in system table?)

  • Save lastID of table on disk, load it then.

  • Scheduler load fix and commit

  • Index SC { delete unused strings, find holes in that we can write strings, write them to it }

  • SC find bigger deleted holes to write data

  • !! BUGFIX deletes BufBlock and BFIle

  • for very big files(>2gbi) do _fseeko64() instead of fseek, fopen64() instead of fopen()

  • properly handle size of buf array > size of buffer (BFile TODO)

  • fbuffer flush only if page is dirty (modified)

  • Clean everything after pftest

  • BFile perf stats collection and print

  • fcl OpenHashMap rehash() fn, also extract and insert

  • PLRUm BBlockManager

  • Index StringColumns

  • StringColumns should be grouped by length in blocks which is hashed by first n letters?

  • Index tables by every column for fast search

  • proper string deleting and indexing? (index cmd to scan all scs and delete unused/doubling, also a scheduled configurable timetable)

  • New string search

  • Fill up scheduler

  • Auto compress logs and auto delete logs

  • Reload Command for reloading DBMS and other stuff

  • A lot of type conversions that cut data available size

  • Proper configs, config manager?

  • https://learn.microsoft.com/en-us/windows/console/reading-and-writing-blocks-of-characters-and-attributes?redirectedfrom=MSDN for normal console output and \r

  • Concurrency on write, normal concurrency

  • Подобие транзакций (при получении запроса, записать его, а потом записать выполнился ли он)

  • RunQuery() function(to handle all the performance and log everything)

  • Add queries to CLI

  • binIO client auto-reading with callbacks or so

  • binIO server side caching

  • Cryptography

  • Migration

  • Flashing

  • Sharding

Ideas:

  • Align everything to 64 to exit false sharing

  • StringColumn for every column of table not all db?

  • Maybe not send table id every query, just select it once and change when you need. save 4b for every select/update/delete packet(query)

  • Client & Server or sock buffering to send a lot of packets at once

  • Write logs for every component on its own

  • Setting to refill deleted ids

  • Unsafe insert?????

  • Somehow align data in tables? (e.g. save all the sizeable and booleans at the end of row)

  • Boolean support?

Bugs:

  • Old bug in binaryIO that causes some packets to be lost and trash to be received [SOLVED] {due to tcp packet maximum size & internal socket buffer, some data was lost}
  • Memory leaks
  • Non-blockable sockets goes wrong data, idk [TEMP-SOLVE] {just use sync sockets for now}
  • select_all binio does not work [SOLVED?] {was sending wrong packet id :| }

BinaryIO:

  • Perf data collection
  • Client side socket buffer
  • Caching
  • Concurrency
  • Connectors
  • GUI client
  • cluster spreading & sharding & replication