MDEV-37949: Implement innodb_log_archive_file_size, innodb_log_archive_path, … #4405
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
dr-m
force-pushed
the
MDEV-37949
branch
4 times, most recently
from
4a9a384 to
9d14e2c
Compare
…target innodb_log_recovery_start: The checkpoint LSN to start recovery from. This will be useful when recovering from an archived log. innodb_log_recovery_target: The requested LSN to end recovery at. This will be useful when recovering data files that were copied as of a time that is before end of the available log.
TODO: Make innodb_log_archive=ON writes fully work! TODO: Rename log files and set innodb_lsn_archived on checkpoint. TODO: Implement recovery from archived log. The new setting SET GLOBAL innodb_log_archive=ON will enable log archiving as soon as the current ib_logfile0 is about to wrap around. The read-only variable innodb_lsn_archived will reflect the LSN since when a complete InnoDB log archive is available. If it is 0 at startup, it will be recovered from the log files. If innodb_log_archive=OFF, this variable will be adjusted to the latest checkpoint every time a log checkpoint is executed. If innodb_log_archive=ON, the value should not change. When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size will affect subsequently created log files when the file that is being currently written is running out. log_t::archive_new_write(): Create and allocate a new log file, and write the outstanding data to both the current and the new file. log_t::archive_new_mmap(): Create and memory-map a new log file, and update file_size to resize_target. log_t::archive_set_size(): Ensure that resize_target is set for new archived log files. log_t::first_lsn: This should be the start of the current log file, to be consulted in log_t::write_checkpoint() when renaming files. log_write_buf(): Add the parameter max_length, the file wrap limit. mtr_t::finish_writer(): Specialize for innodb_log_archive=ON innodb_log_archive_update(): Assign log_sys.archived_lsn to the latest checkpoint; it should be usable. If the log buffer is about to wrap around, back off and retry. log_t::archived_mmap_switch_prepare(): Create and memory-map a new log file. TODO: Do we need a separate function from append_prepare_wait()? log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case. log_t::archived_mmap_switch_complete(): Switch to the buffer that was created in archived_mmap_switch_prepare(). mtr_t::finish_writer(): Invoke log_close() only if innodb_log_archive=OFF.
innodb.log_file_size_online: Add a restart in order to preserve the innodb_lsn_archived after running with innodb_log_archive=OFF. Failing tests with the log_t::ARCHIVED_MMAP code path (PMEM): mysql-test/mtr --mysqld=--innodb-log-archive --parallel=auto \ --force --big-test --max-test-fail=0 \ encryption.innochecksum \ encryption.innodb-page_encryption-32k \ innodb.alter_copy_bulk \ innodb.bulk_load \ innodb.innodb-index \ innodb.innodb_bulk_create_index_debug \ innodb.insert_into_empty \ innodb.shrink_cached_undo \ innodb.sys_truncate_debug \ innodb.sys_truncate_large \ innodb.sys_truncate_shutdown_debug \ innodb.undo_truncate \ innodb.undo_truncate_recover \ innodb_fts.bug_32831765 \ innodb_gis.rtree_split \ main.analyze_engine_stats2 \ main.analyze_stmt_prefetch_count \ mariabackup.huge_lsn \ mariabackup.mdev-14447 \ mariabackup.undo_truncate \ stress.ddl_innodb Note: With the pwrite(2) based code path, innodb_log_archive=ON is totally broken.
log_t::set_archive(my_bool): Implement SET GLOBAL innodb_log_archive. An error will be returned if non-archived SET GLOBAL innodb_log_file_size (log file resizing) is in progress. FIXME: Rename and update the current log file. innodb_log_archive_start: The initial value of the status variable innodb_lsn_archived, or 0 to autodetect from the log files.
log_t::set_archive(my_bool): Implement SET GLOBAL innodb_log_archive. The log file will be renamed. log_t::header_rewrite(my_bool): Rewrite the log file header before or after renaming the log file. The recovery of the last archived log file must tolerate also the ib_logfile0 format. TODO: When starting up with innodb_log_archive=ON, create the log file in that format, using an appropriate name. TODO: Throw an error on startup if the innodb_log_archive setting does not match the log file name (and contents). TODO: Implement crash recovery for the archived-format log file. TODO: Use a different file name for encrypted archived log.
recv_sys_t::find_checkpoint(): If the circular ib_logfile0 is missing, determine the oldest archived log file with contiguous LSN. For now, we open the first archived log file. FIXME: Validate innodb_log_archive_start, innodb_log_recovery_start, innodb_log_recovery_target FIXME: In bootstrap, write the header in the correct format. FIXME: Parse the archived log file header in the correct format. FIXME: Multi-file recovery
log_t::write_checkpoint(): Write checkpoint in the archived format (almost). log_t::create(): In the archived format, let the header remain zero-filled. log_t::set_archive(): Fix some possible races
buf_dblwr_t::create(): Create the doublewrite buffer in a single atomic mini-transaction. Do not write any log records for initializing any doublewrite buffer pages, in order to avoid recovery failure with innodb_log_archive=ON starting from the very beginning.
Initial, crude version of innodb_log_archive=ON recovery. log_t::attach(): For memory-mapped log, store the file handle in resize_log so that the file can be made read-only in write_checkpoint(). recv_sys_t::find_checkpoint(): Do not read any ib_logfile* if innodb_log_archive=ON. recv_sys_t::parse(), log_parse_file(): Tweak some logic for innodb_log_archive=ON.
log_t::persist(): Even if the flushed_to_disk_lsn does not change, we may want to reset the write_lsn_offset.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
TODO: fill description here
Release Notes
TODO: What should the release notes say about this change?
How can this PR be tested?
TODO: modify the automated test suite to verify that the PR causes MariaDB to behave as intended.
Basing the PR against the correct MariaDB version
mainbranch.This is a new feature, but for now based on the 11.4 branch so that any unrelated errors that may be found during testing can be fixed rather quickly. Merges to the
mainbranch may be blocked for weeks at a time.PR quality check