cc @taigrr
This PR will _hopefully_ help to fix some critical isseus in the real world with several or more [Yarn.social](https://yarn.social) pods running [yarnd](https://git.mills.io/yarnsocial/yarn) where starting back up after a power failure or crash can sometimes result in an empty `config.json` or empty `meta.json` or both!
I'm not actually sure how this can arise, and as yet I haven't been able to reproduce it (_I can only assume this has to be failures cases outside of our control_); but in any case the application and database is recoverable by simply `rm config.json` and/or `rm meta.json`.
So this PR makes errors loading the config and metadata first-class errors and exported error types that consumers of the library can use to perform automated recovery without requiring human intervention.
Basiclaly in this case it's no big deal we lost the database config of metadata, we can simply carry on.
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/241
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
Fix issues related to #239
Disable mmap reader for current datafile, which only read from the fd.
Co-authored-by: jason3gb <jason3gb@gmail.com>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/240
Reviewed-by: James Mills <james@mills.io>
Reviewed-by: Tai Groot <tai@taigrr.com>
Co-authored-by: jason3gb <jason3gb@noreply@mills.io>
Co-committed-by: jason3gb <jason3gb@noreply@mills.io>
Related to #234 and !236.
This is the implementation that was requested in the original issue. I updated KEYS command to be redis-valid and implemented prefix search. There is also a rather interesting test, I could you use some feedback here.
I noticed that it might not be possible to reduce the complexity of the KEYS command. Because even if you use Scan, you will have to store the counter of all found keys before you do WriteBulk of the actual keys.
@prologic here is what you probably had in mind:
```
s.db.Scan([]byte(prefix), func(key []byte) error {
conn.WriteBulk(key)
return nil
})
```
But there is no way to call `conn.WriteArray(n)` with the number of keys until you iterate through all of them, hence the second loop over found keys.
Co-authored-by: Ivan Elfimov <ielfimov@gmail.com>
Co-authored-by: James Mills <james@mills.io>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/237
Reviewed-by: James Mills <james@mills.io>
Co-authored-by: biozz <biozz@noreply@mills.io>
Co-committed-by: biozz <biozz@noreply@mills.io>
Co-authored-by: James Mills <1290234+prologic@users.noreply.github.com>
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Co-authored-by: Tai Groot <tai@taigrr.com>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/160
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
Added Sift and ScanSift functions for review without tests (for now)
fix docstrings
Added tests for Sift and ScanSift
Note this also fixes a bug in the Scan() function where the RMutex is not locked, allowing a potential race condition
closes#231
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/232
Co-authored-by: Tai Groot <tai@taigrr.com>
Co-committed-by: Tai Groot <tai@taigrr.com>
Fixes#228
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/229
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
[ADDED] new tests for TTL expiration race condition, see #216
[REMOVED] removes cleanup / automatic expiration from get() function to resolve#216
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/227
Co-authored-by: Tai Groot <tai@taigrr.com>
Co-committed-by: Tai Groot <tai@taigrr.com>
Supercesd #219 after rebasing on master after migrating off Github.
Co-authored-by: Nicolò Santamaria <nicolo.santamaria@protonmail.com>
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Co-authored-by: Tai Groot <taigrr@noreply@mills.io>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/224
Co-authored-by: James Mills <prologic@noreply@mills.io>
Co-committed-by: James Mills <prologic@noreply@mills.io>
Fixes#222
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/223
Co-authored-by: James Mills <prologic@noreply@mills.io>
Co-committed-by: James Mills <prologic@noreply@mills.io>
* Add test case for Locking after Merge
* retain lock file after merge
* remove replacing lock file (not needed)
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Co-authored-by: yash <yash.chandra@grabpay.com>
