Commit Graph
858 Commits
Author SHA1 Message Date
Sergei TrifonovandGitHub c10e42ca05 Merge branch 'master' into async-loader-integration 2023-08-28 20:20:31 +02:00
Robert Schulze f2fb7365e6 Add settings for 1. size_ratio (all caches) and 2. policy (index caches)
1. is an expert-level setting, default is 0.5, applies only to SLRU.

Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
2023-08-22 11:04:29 +00:00
Robert SchulzeandGitHub 583d9b33f0 Merge pull request #51446 from rschu1ze/recfgable-caches
Runtime-reconfigurable cache sizes
2023-08-21 23:27:21 +02:00
Nikita MikhaylovandGitHub 145c99ee94 Merge pull request #53280 from evillique/better-stop-listen
Add EXCEPT clause to SYSTEM STOP LISTEN query
2023-08-21 19:03:49 +02:00
Robert Schulze 96c8b250ce Allow to configure cache sizes at runtime 2023-08-21 13:51:10 +00:00
kssenii 5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
robot-ch-test-pollandGitHub ce58b90ea1 Merge pull request #53455 from rschu1ze/respect-changed-cgroups-limit
Respect cgroup memory limit when reloading configuration
2023-08-18 00:03:35 +02:00
Alexander TokmakovandGitHub 89c365803f Merge pull request #53424 from ClickHouse/exception_message_patterns6
Less exceptions with runtime format string
2023-08-17 17:58:04 +03:00
Robert Schulze 5253a7f650 Respect cgroup memory limit when reloading configuration
Cgroups allows to change the amount of memory available to a process
while it runs. The previous logic calculated the amount of available
memory only once at server startup. As a result, memory thresholds set
via cgroups were not picked up when the settings changed. We now always
incorporate the current limits during re-configuraton.

Note 1: getMemoryAmount() opens/reads a file which is potentially
        expensive. Should be fine though since that happens only when
        the server configuration changes.

Note 2: An better approach would be to treat cgroup limit changes as
        another trigger for ClickHouse server re-configuration (which
        currently only happens when the config files change). Shied away
        from that for now because of the case that when the cgroup limit
        is lowered, there is no guarantee that ClickHouse can shrink the
        memory amount accordingly in time (afaik, it does so only lazily
        by denying new allocations). As a result, the OOM killer would
        kill the server. The same will happen with this PR but at a
        lower implementation complexity.
2023-08-17 09:22:13 +00:00
Nikolay Degterinsky a893f75884 Merge remote-tracking branch 'upstream/master' into better-stop-listen 2023-08-16 23:06:18 +00:00
Alexander Tokmakov 894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Robert Schulze 7ed5166c8a Cosmetics: Make re-cfg of query cache similar to re-cfg of other caches
This doesn't change semantics, it only changes in which places the
configuration is parsed. This makes the logic more consistent across
caches.
2023-08-14 15:42:32 +00:00
Robert Schulze f126e54b0d Respect max_cache_size for all caches
So far, max_cache_size was only respected in Server.cpp, and in this
file only for the mark cache and the uncompressed cache. This commit
respects it also for the other cache types in Server.cpp, as well for
all caches in LocalServer.cpp.
2023-08-14 15:42:32 +00:00
Robert Schulze 54e54aed74 Cosmetics: Introduce constants for default cache sizes and policies 2023-08-14 15:42:32 +00:00
kssenii c3b8bc0267 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-13 21:14:42 +04:00
Kseniia SumarokovaandGitHub ceb74ad3cb Merge branch 'master' into allowed-caches-dir-for-dynamic-disks 2023-08-11 16:49:02 +02:00
Alexey Milovidov a55f9eae84 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-creating-system-tables-at-startup 2023-08-11 14:45:03 +02:00
Nikolay Degterinsky bcc0fbbf91 Add EXCEPT clause to SYSTEM STOP LISTEN query 2023-08-10 16:10:33 +00:00
kssenii 2cfb4c0264 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-09 11:09:34 +02:00
Alexey Milovidov fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Alexey Milovidov e1206dbaab Add a comment from @tavplubix 2023-08-07 18:57:30 +02:00
kssenii 0c174a7ec1 Allow to restrict fs caches paths 2023-08-07 18:00:26 +02:00
Alexey Milovidov 167f90e7f6 Maybe better 2023-08-06 13:28:21 +02:00
Sergei TrifonovandGitHub 432d359a2b Merge branch 'master' into async-loader-integration 2023-08-04 18:51:06 +02:00
Alexander Tokmakov 67d8f1c1c5 fix data race 2023-08-02 18:42:38 +02:00
Sergei TrifonovandGitHub e4ae2ece7c Merge branch 'master' into async-loader-integration 2023-08-01 13:04:11 +02:00
Alexey MilovidovandGitHub a7b4a29ed9 Merge branch 'master' into fixes-start-stop-listen 2023-07-31 08:34:17 +03:00
alesapinandGitHub aba291ca68 Revert "Revert "Add an ability to specify allocations size for sampling memory profiler"" 2023-07-30 14:01:34 +02:00
Alexey MilovidovandGitHub 5d2fb786f6 Merge branch 'master' into fixes-start-stop-listen 2023-07-29 00:39:58 +03:00
pufitandGitHub 6211845ef0 Merge pull request #51303 from ClickHouse/deprecate-metadata-cache
Deprecate the `metadata_cache` feature
2023-07-28 10:31:02 -04:00
Nikolay Degterinsky b5160c8072 Fix bugs and better test for SYSTEM STOP LISTEN 2023-07-27 14:16:19 +00:00
d89e2e6a27 Add SYSTEM STOP LISTEN query (#51016)
Co-authored-by: Nikita Mikhaylov <[email protected]>
Co-authored-by: Nikita Mikhaylov <[email protected]>
2023-07-26 19:58:41 +02:00
Alexander TokmakovandGitHub 6bbed6262e Merge branch 'master' into add_delay_for_replicated 2023-07-26 12:48:48 +03:00
Anton PopovandGitHub 4531b105ac Merge branch 'master' into flush-async-insert-queue 2023-07-25 15:48:01 +02:00
Alexander Tokmakov faca49a905 Merge branch 'master' into add_delay_for_replicated 2023-07-24 16:07:38 +02:00
Anton Popov 347f442f84 Merge remote-tracking branch 'upstream/master' into HEAD 2023-07-24 10:55:31 +00:00
Alexey Milovidov a17652a422 Merge branch 'master' into incbin 2023-07-24 05:33:52 +02:00
robot-ch-test-poll4andGitHub fb045b0d86 Merge pull request #52498 from ClickHouse/dont_shutdown_interserver_before_tables
Don't shutdown interserver before tables
2023-07-24 01:45:25 +02:00
Alexey Milovidov 1e467867e6 Attempt to fix LTO 2023-07-24 00:03:40 +02:00
alesapin 753a8d0a43 Merge branch 'dont_shutdown_interserver_before_tables' into add_delay_for_replicated 2023-07-23 22:41:15 +02:00
alesapin e02948580b Don't shutdown interserver before tables 2023-07-23 22:38:59 +02:00
Alexey MilovidovandGitHub 43bd6d1b83 Revert "Add an ability to specify allocations size for sampling memory profiler" 2023-07-23 19:00:49 +03:00
Alexey Milovidov 00d6f2ee08 Use incbin for resources, part 1 2023-07-23 06:11:03 +02:00
alesapinandGitHub d1fa105d60 Merge branch 'master' into add_delay_for_replicated 2023-07-23 00:12:29 +02:00
alesapinandGitHub 835c36cbf3 Merge pull request #52419 from ClickHouse/implement_borders_for_sampling_memory_profiler
Add an ability to specify allocations size for sampling memory profiler
2023-07-22 18:13:26 +02:00
alesapinandGitHub 6416fb6eed Merge branch 'master' into add_delay_for_replicated 2023-07-22 12:11:39 +02:00
Alexander Sapin 4695ec6802 Add an ability to specify allocations size for sampling memory profiler 2023-07-21 14:56:29 +02:00
Dmitry Kardymon 5124f74fcc Merge remote-tracking branch 'origin/master' into ADQM-996 2023-07-21 05:54:36 +00:00
alesapin cbcd48979c Fix race one more time 2023-07-20 13:04:43 +02:00
serxa 02fc66db10 avoid using bg pool during non-async start 2023-07-19 17:37:53 +00:00