Commit Graph

185 Commits (master)

Author SHA1 Message Date
Edward Shen 53015e116f
Renamed EncryptedDiskReader to EncryptedReader 2021-07-14 13:32:26 -04:00
Edward Shen 7ce974b4f9
Make EncryptedDiskReader generic over R 2021-07-14 13:32:00 -04:00
Edward Shen 973ece3604
MetadataFuture tests, fix UB 2021-07-14 13:28:09 -04:00
Edward Shen 0c78b379f1
Remove comments w.r.t. potential db optimizations
DB queries already are performed on their own thread, so spawning a
thread to do db work is unncessary.
2021-07-14 11:21:28 -04:00
Edward Shen 94375b185f
More debugging 2021-07-13 23:12:29 -04:00
Edward Shen 6ac8582183
Encrypted files work in debug mode 2021-07-13 20:38:01 -04:00
Edward Shen 656543b539
more partial work into encryption 2021-07-13 16:39:32 -04:00
Edward Shen 2ace8d3d66
Partial rewrite of encrypted writer 2021-07-13 13:16:44 -04:00
Edward Shen 160f369a72
Migrate to tracing crate 2021-07-12 23:23:51 -04:00
Edward Shen 9f76a7a1b3
Add compression middleware 2021-07-12 16:39:06 -04:00
Edward Shen 868278991a
Add disk tests 2021-07-12 15:59:52 -04:00
Edward Shen e8bea39100
Remove serialize impl from legacy structs 2021-07-12 13:43:47 -04:00
Edward Shen acc3ab2186
use hex instead of u8s in test 2021-07-12 00:49:06 -04:00
Edward Shen 8daa6bdc27
add tests for Md5Hash conversions 2021-07-12 00:48:00 -04:00
Edward Shen 69587b9ade
Clippy lints 2021-07-12 00:12:15 -04:00
Edward Shen 8f3430fb77
Add support for reading old db image ids 2021-07-11 23:33:22 -04:00
Edward Shen ec9473fa78
clippy lint 2021-07-11 23:25:17 -04:00
Edward Shen 3764af0ed5
Optimize header creation 2021-07-11 14:23:15 -04:00
Edward Shen 099c795cca
Add potential perf gains in the future 2021-07-11 14:22:59 -04:00
Edward Shen 92a66e60dc
Seek file from beginning on encrypted header 2021-07-11 13:25:02 -04:00
Edward Shen 5143dff888
Add logging 2021-07-11 13:21:57 -04:00
Edward Shen 7546948196
nightly clippy lints 2021-07-11 13:19:37 -04:00
Edward Shen 5f4be9809a
Add support for legacy files 2021-07-11 02:33:51 -04:00
Edward Shen 8040c49a1e
Add warning for encryption 2021-07-11 00:15:43 -04:00
Edward Shen f64d03493e
use static default headers 2021-07-10 19:04:27 -04:00
Edward Shen 93249397f1
Simply codebase 2021-07-10 18:53:28 -04:00
Edward Shen 154679967b
writing optimizations 2021-07-10 14:22:29 -04:00
Edward Shen b90edd72a6
Add clippy to CI 2021-07-09 21:25:08 -04:00
Edward Shen 3a855a7e4a
add debug for client config 2021-07-09 19:51:48 -04:00
Edward Shen 5300afa205
Clean up ClientSecret usage 2021-07-09 19:48:25 -04:00
Edward Shen 88561f7c2c
Fix short arg conflict 2021-07-09 19:17:56 -04:00
Edward Shen ce03ce0baf
add support for yaml files 2021-07-09 19:14:53 -04:00
Edward Shen a8e5d09ff0
clippy lints 2021-07-09 17:20:15 -04:00
Edward Shen b4f27c5f8c
migrate config to Config struct 2021-07-09 17:18:43 -04:00
Edward Shen 5b67431778
initial work 2021-07-09 14:36:04 -04:00
Edward Shen fe967a2272
clippy 2021-06-24 10:41:42 -04:00
Edward Shen 98ab7ab9e8
Blackhole non-token routes 2021-06-24 10:39:12 -04:00
Edward Shen 51173fa1a3
update index.html 2021-06-06 20:36:51 -04:00
Edward Shen 2501bb8da0
Upstream client to always use HTTP/2 and HTTPS 2021-06-06 18:51:05 -04:00
Edward Shen 97ea5eb448
Use short SHA in start message 2021-06-06 18:17:42 -04:00
Edward Shen 75752eb13e
remove unnecessary negation 2021-06-06 18:03:47 -04:00
Edward Shen 85c644ffed
Add git commit version 2021-06-06 17:48:48 -04:00
Edward Shen 9e9c955f5a
Update client spec version 2021-05-28 15:30:25 -04:00
Edward Shen 1c8f3da8d0
More logging 2021-05-27 19:24:54 -04:00
Edward Shen 5536f9638a
Add index page 2021-05-27 17:05:50 -04:00
Edward Shen 00739ec645
More logging 2021-05-27 16:47:35 -04:00
Edward Shen 491d0c9fda
Fix reading from encrypted files 2021-05-27 16:22:32 -04:00
Edward Shen f038452263
Fix logging messages 2021-05-27 16:22:15 -04:00
Edward Shen c32e534300
Revamp reading potentially encrypted files
Previously, we'd assume that if the header was successfully parsed, then
we'd have a valid encrypted header. This isn't the case, and now instead
we try to read metadata, first unencrypted, then encrypted, to make sure
we're reading with the correct abstraction.
2021-05-27 16:19:26 -04:00
Edward Shen a732019f60
use split_at for token parsing 2021-05-24 15:21:45 -04:00
Edward Shen b8c12b463f
Use macro for metrics generation 2021-05-23 19:05:57 -04:00
Edward Shen 579d316945
clippy 2021-05-22 23:10:34 -04:00
Edward Shen 5fd8e86d97
Add support for offline mode 2021-05-22 23:06:05 -04:00
Edward Shen 79f73ed68e
Add prometheus endpoint 2021-05-22 22:10:03 -04:00
Edward Shen 96a6b76baa
better combinator 2021-05-22 20:19:17 -04:00
Edward Shen 7acc126de2
optimize file opening 2021-05-20 14:23:16 -04:00
Edward Shen ea8e031a3d
Remove extraneous conversion 2021-05-20 13:49:37 -04:00
Edward Shen 7f16e697e6
Fix lints 2021-05-20 13:42:08 -04:00
Edward Shen a3f3b5e3ab
Gracefully handle secretstream errors 2021-05-20 13:35:57 -04:00
Edward Shen a70f4bfdc3
Support loading disk size from db 2021-05-20 00:38:28 -04:00
Edward Shen 6daadefaf1
Resolve clippy lints 2021-05-19 23:52:15 -04:00
Edward Shen 0fcde518c5
Use qualified instead of absolute 2021-05-19 23:29:18 -04:00
Edward Shen c488eccc8d
Clean up traits 2021-05-19 23:27:56 -04:00
Edward Shen 4f55380d23
Remove layer of indirection 2021-05-19 22:42:44 -04:00
Edward Shen b66cccc832
implement at-rest encryption 2021-05-19 21:42:55 -04:00
Edward Shen 8772df2231
Add additional handling for ping errors 2021-05-19 20:33:39 -04:00
Edward Shen fe28f19f07
initial work into encryption at rest 2021-05-11 21:01:01 -04:00
Edward Shen eb605b9c27
merge memory cache types 2021-05-11 18:56:00 -04:00
Edward Shen 739137c4be
Fixed erronous log statement 2021-05-11 16:56:28 -04:00
Edward Shen 6da2cba78a
Use unstable options 2021-04-25 12:55:31 -04:00
Edward Shen 0857ffadc7
Add LFU cache 2021-04-24 23:56:56 -04:00
Edward Shen 0db06fcabd
explicitly mem drop lazy init failure 2021-04-24 12:57:32 -04:00
Edward Shen 4a36541570
reorder imports 2021-04-24 12:47:33 -04:00
Edward Shen 151d0c5a83
Add docs 2021-04-24 12:46:18 -04:00
Edward Shen 9abeec89bc
use arc_swap for tls 2021-04-24 00:56:58 -04:00
Edward Shen 70015a7c38
comment why tls is slow 2021-04-23 23:28:47 -04:00
Edward Shen 654178dd5c
Use static HTTP client instead 2021-04-23 22:23:24 -04:00
Edward Shen 424dc09ae3
remove more unwraps 2021-04-23 18:25:59 -04:00
Edward Shen eb8a334a05
Remove some unwraps 2021-04-23 18:03:53 -04:00
Edward Shen 141b5b257c
rename lowmem cache to disk cache 2021-04-23 17:22:29 -04:00
Edward Shen 0918b210ea
don't fill logs with compromised response 2021-04-23 00:34:23 -04:00
Edward Shen ef5ed89626
add missing files 2021-04-23 00:13:36 -04:00
Edward Shen d9cebcbb4b
Add memory accelerated disk cache 2021-04-23 00:11:30 -04:00
Edward Shen 77cd416849
Move task to separate function 2021-04-22 21:55:26 -04:00
Edward Shen a86cd3edf5
Don't use arc for cache keys 2021-04-22 21:46:34 -04:00
Edward Shen 8d95fe3f07
remove unnecessary channel 2021-04-22 21:34:31 -04:00
Edward Shen 758b0ec78d
Fix updating db timestamp 2021-04-22 21:29:26 -04:00
Edward Shen 288872e84b
Add db queries 2021-04-22 21:21:32 -04:00
Edward Shen b01fa618b4
Add buffered reader again 2021-04-22 20:01:11 -04:00
Edward Shen 2193d74e24
Fix not skipping over metadata 2021-04-22 19:22:34 -04:00
Edward Shen 8cc21f4803
Revert "use bufreaders"
This reverts commit f335b99024.
2021-04-22 18:09:48 -04:00
Edward Shen f335b99024
use bufreaders 2021-04-22 17:52:16 -04:00
Edward Shen 3c1388fced
false positive 2021-04-22 13:40:19 -04:00
Edward Shen 7e4c54867e
Downgrade sql statements to trace 2021-04-22 13:20:45 -04:00
Edward Shen 6fda24186b
use debug logging level for sql statements 2021-04-22 13:18:50 -04:00
Edward Shen 5099666322
temporary remove generational 2021-04-22 13:11:08 -04:00
Edward Shen eab0449a02
use Arc for cache keys 2021-04-22 13:03:33 -04:00
Edward Shen efd6b5c60c
Remove cache from hot path 2021-04-22 12:44:02 -04:00
Edward Shen 53c0cb664a
fix truncated bytes 2021-04-20 17:48:04 -04:00
Edward Shen 8f03aa0236
use cleaner concurrentfs implementation 2021-04-20 17:28:02 -04:00