1
0
mirror of https://github.com/taigrr/bitcask synced 2025-01-18 04:03:17 -08:00

45 Commits

Author SHA1 Message Date
d3428bac8c
Drop support for Windows (Closes #192) 2020-12-11 20:33:53 +10:00
yashschandra
158f6d9888
Get space that can be reclaimed (#189)
* get reclaimable space added

* import order fix

Co-authored-by: yash <yash.chandra@grabpay.com>
2020-12-01 06:07:00 +10:00
yashschandra
f4357e6f18
local live backup support (#185)
* live backup first commit

* exclude lock file in backup

* create path if not exist for backup

Co-authored-by: yash <yash.chandra@grabpay.com>
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
2020-11-30 07:49:02 +10:00
5e01d6d098
Add a few more test cases for concurrent operations 2020-11-27 16:52:08 +10:00
Bryan Stenson
295301a44c
Add configuration options for FileMode (#183)
* Add configuration options for FileMode

Add two additional configuration values, and their corresponding default values:

* DirFileModeBeforeUmask - Dir FileMode is used on all directories created.  DefaultDirFileModeBeforeUmask is 0700.
* FileFileModeBeforeUmask - File FileMode is used on all files created, except for the "lock" file (managed by the Flock library).  DefaultFileFileModeBeforeUmask is 0600.

When using these bits of configuration, keep in mind these FileMode values are set BEFORE any umask rules are applied.  For example, if the user's umask is 022, setting DirFileFileModeBeforeUmask to 777 will result in directories with FileMode set to 755 (this umask prevents the write bit from being applied to group and world permissions).

* moving defer statements after checking for errors

use os.ModePerm const instead of os.FileMode(777)

* fix spelling/grammar

* skip these tests for Windows as they appear to break - Windows is less POSIX-y than it claims

* ignore "lock" file for default case too -- this was incorrectly passing before including this, as my local dev station has umask 022
2020-11-05 08:06:45 +10:00
Ignacio Hagopian
8dca9cd2a7
Auto recovery (#153)
* implement autorepair

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* fix misspell

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* Update internal/data/recover.go

Co-authored-by: James Mills <prologic@shortcircuit.net.au>

* Update internal/utils.go

Co-authored-by: James Mills <prologic@shortcircuit.net.au>

* Update internal/data/recover.go

Co-authored-by: James Mills <prologic@shortcircuit.net.au>

* skip failing test on windows

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

Co-authored-by: James Mills <prologic@shortcircuit.net.au>
2020-05-08 03:48:36 +10:00
Ignacio Hagopian
7b24d87695
don't allow empty keys (#151)
* bitcask: don't allow empty keys

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>

* go mod tidy

Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-05-06 10:47:41 +10:00
Ignacio Hagopian
17e96a1a45
tests: fix wg race (#150)
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-04-20 11:05:07 +10:00
James Mills
45c81452c5
Added test harness logic to skip some tests on Windows (#129) 2020-01-26 07:31:58 +10:00
Alain Gilbert
ca06e332d6 Add DeleteAll function (#116) 2019-12-23 21:35:59 +10:00
Alain Gilbert
be3fd71ebe Fix loadIndex to be deterministic (#115) 2019-12-20 14:45:10 +10:00
Leonid Zharikov
4dfe42cb3b Export method reopen (#113) 2019-11-16 21:08:45 +10:00
James Mills
65e9317d26
Fix glfmt/golint issues 2019-10-14 16:55:47 +10:00
James Mills
5ea05fb3c2
Add unit test for opening bad database with corrupted/invalid datafiles (#105) 2019-09-26 21:07:13 +10:00
Ignacio Hagopian
5be114adab
Makefile setup & key/value coherent datatypes & refactoring (#98)
* internal/data: comment exported functions

* internal/data: make smaller codec exported api surface

* make key and value sizes serializing bubble up to everything

* Makefile setup & go mod tidy
2019-09-12 10:44:26 -03:00
James Mills
d59d5ad8c2
Improves Test Coverage by covering error cases (#95)
* Add Unit  Test for testing a corrupted config

* Add Unit Test for testing errors from .Stats()

* Refactor  Datafile into an interface and add Unit Tests for testing Merge() errors

* Refactor indexer into an interface and add Unit Tests for .Close() errors

* Add Unit Tests for .Delete() errors

* Add Unit Tests for  testing Put/Get errors

* Add Unit Test for testing Open errors (bad path for example)

* Refactor out bitcask.writeConfig

* Add more tests for config errors

* Add unit test for options that might error

* Add more test cases for close errors

* Add test case for rotating datafiles

* Fix a possible data race in .Stats()

* Add test case for checksum errors

* Add test case for Sync errors with Put and WithSync enabled

* Refactor and use testify.mock for mocks and generate mocks for all interfaces

* Refactor TestCloseErrors

* Refactored TestDeleteErrors

* Refactored TestGetErrors

* Refactored TestPutErrors

* Refactored TestMergeErrors and fixed a bug with .Fold()

* Add test case for Scan() errors

* Apparently only Scan() can return nil Node()s?
2019-09-09 07:18:38 +10:00
James Mills
f4fb4972ee
Improves test coverage by adding some missing unit tests (#90)
* Add Unit Test for testing WithSync() option

* Add Unit Test for testing re-indexing

* Add Unit Test for testing re-indexing with deleted keys (tombstone values)
2019-09-04 22:45:04 +10:00
James Mills
c5a565cd82
Adds WithSync(...) option to turn on sync after write durability (#63)
* Added WithSync(...) option to turn  on sync after write durability

* Add Sync/NoSync benchmark variants for Put()
2019-08-12 06:47:46 +10:00
Ignacio Hagopian
fd179b4a86 custom high-performance encoder implementation (#52) 2019-08-08 09:21:46 +10:00
James Mills
755b1879b5
Use []byte byte slices as keys directly avoiding serialing string(s) (#46) (#51) 2019-08-08 08:14:48 +10:00
James Mills
d0c913ccee
Revert "Use []byte byte slices as keys directly avoiding serialing string(s) (#46)" (#50)
This reverts commit 3c1808cad3f19c23c6e4aacd4cfbbc4a04da1c08.
2019-08-08 08:06:38 +10:00
James Mills
3c1808cad3
Use []byte byte slices as keys directly avoiding serialing string(s) (#46) 2019-08-08 07:59:11 +10:00
Ignacio Hagopian
a407905ae2 Improve Get/Put performance with optional mempooling (#36)
* avoid unnecessary use of encoder/decoder to decrease memory allocations

* add an optional configurable mempool to avoid extra allocs

* add doc.go with examples
2019-08-05 07:23:07 +10:00
James Mills
51bac21c0a
Improves Merge() operation by also pruning old key/value pairs (#29)
* Added new API Stats() and Prune()

* Improved Merge() logic to also prune old key/values and actually reclaim disk space

* Added backward compat for the old Merge() function

* Refactor indexing of keys to items (hints)

* Remove redundant TestOpenMerge

* Add unit test for Stats()

* Improve TestMerge()
2019-07-27 07:52:25 +10:00
Awn
c28c72108f Use consistent directory names in benchmarks (#24)
* patch: use current directory for benchmarks

* Use consistent directory names
2019-07-25 08:18:41 +10:00
Awn
a74203b99e patch: use current directory for benchmarks (#23) 2019-07-25 08:09:04 +10:00
Awn
3b63388e79 benchmarks: set test size to compute throughput (#16)
* benchmarks: set test size to compute throughput

* patch: error on cross-filesystem rename operation

* revert: commit to wrong branch
2019-07-25 07:14:58 +10:00
Christian Muehlhaeuser
057c147f89 Added missing error check (#13)
err was assigned, but not checked.
2019-07-19 20:22:56 +10:00
James Mills
352c32ee12
Add Len() to exported API (extended API) 2019-03-21 10:47:50 +10:00
James Mills
aaea7273c3
Add Keys() to exported API (extended API) 2019-03-21 10:41:56 +10:00
James Mills
962e53af17
Add Has() to exported API (extended API) 2019-03-21 10:24:48 +10:00
James Mills
8bf169c96f
Add MergeOpen test case 2019-03-20 17:10:24 +10:00
James Mills
c1488fed2a
Added Fold() test case 2019-03-20 16:55:59 +10:00
James Mills
d8a48f9eea
Use pre-defined errors as they are comparable and useful as exported symbols 2019-03-20 07:39:03 +10:00
James Mills
cb00b11dd7
Increase no. of goroutines to catch more race conditions in tests 2019-03-16 12:33:07 +10:00
James Mills
120e854444
Improved error messages 2019-03-16 11:47:22 +10:00
James Mills
3f1d6635c4
Add prefix scan for keys using a Trie 2019-03-15 23:48:50 +10:00
James Mills
d24a01797a
Added WithMaxKeySize() and WithMaxValueSize() options 2019-03-14 21:31:23 +10:00
James Mills
bc8f6c6718
Change locking error message 2019-03-14 21:31:01 +10:00
James Mills
71a42800fe
Improved benchmark test suite for various key/value sizes 2019-03-14 18:17:20 +10:00
James Mills
3b9627aeb8
Fix concurrent read bug 2019-03-14 17:58:06 +10:00
James Mills
e0c4c4fdae
Fix concurrent write bug with multiple goroutines writing to the to the active datafile 2019-03-14 17:58:06 +10:00
James Mills
fb2335e3c1
Fixed tests 2019-03-14 07:46:59 +10:00
James Mills
f4b7918e93
Add flock on database Open()/Close() to prevent multiple concurrent processes write access. Fixes #2 2019-03-13 20:21:15 +10:00
James Mills
a42cd20ddf
Initial Commit 2019-03-12 23:56:57 +10:00