Investigating memory consumption #3658
Replies: 2 comments 6 replies
-
|
I would be nice to know more about your script to be able to help more.. |
Beta Was this translation helpful? Give feedback.
-
def playlist.manager(~id=null())
id = string.id.default(id, default="playlist")
log = logger(id)
playlists = ref([])
def load_playlist(path)
_playlist = playlist(path, mode="randomize", reload_mode="watch")
id = _playlist.id()
playlists := list.add((id, _playlist), playlists())
_playlist
end
def get_playlists()
playlists()
end
get_playlists.{load = load_playlist}
end
security = single("/var/liquidsoap/security.ogg")
playlists = playlist.manager()
jingles = playlists.load("/media/jingles")
music_A = playlists.load("/media/music/A")
music_B = playlists.load("/media/music/B")
# 40 more playlists here
music_Y = playlists.load("/media/music/Y")
music_Z = playlists.load("/media/music/Z")
monday_morning = random([music_A, music_Z], weights=[20, 10])
monday_afternoon = random([music_B, music_Y], weights=[20, 10])
# more random function calls here
sunday_evening = random([music_S, music_O], weights=[10, 20])
sunday_night = random([music_F, music_Q], weights=[20, 10])
schedule = switch([
({ 1w6h-1w12h }, monday_morning),
({ 1w12h-1w16h }, monday_afternoon),
# schedule for the full week
({ 7w19h-7w20h }, sunday_evening),
({ 7w20h-1w6h }, sunday_night),
])
rotation = rotate(weights = [3, 1], [schedule, jingles])
radio = switch(track_sensitive = false, [(schedule.is_ready, rotation), ({true}, security)])
radio.on_metadata(print)
output.dummy(radio)Also def api.v1.default()
port = api.v1.http_port
path = regexp("^#{api.v1.http_path_prefix}/")
handler = fun(_) -> http.response.json.fail()
harbor.http.register.simple.regexp(path, handler, method="GET", port=port)
harbor.http.register.simple.regexp(path, handler, method="POST", port=port)
end
def api.v1.server.register_commands()
path_prefix = "#{api.v1.http_path_prefix}/server"
port = api.v1.http_port
base = api.v1.server
get_endpoint = fun(path, handler) -> harbor.http.register.simple(path, handler, method="GET", port=port)
post_endpoint = fun(path, handler) -> harbor.http.register.simple(path, handler, method="POST", port=port)
get_endpoint("#{path_prefix}/cpu", base.cpu)
get_endpoint("#{path_prefix}/memory", base.memory)
post_endpoint("#{path_prefix}/restart", base.restart)
gc_path_prefix = "#{path_prefix}/gc"
gc_base = api.v1.server.gc
get_endpoint("#{gc_path_prefix}", gc_base.get)
post_endpoint("#{gc_path_prefix}", gc_base.set)
get_endpoint("#{gc_path_prefix}/stat", gc_base.stat)
get_endpoint("#{gc_path_prefix}/quick_stat", gc_base.quick_stat)
post_endpoint("#{gc_path_prefix}/compact", gc_base.compact)
post_endpoint("#{gc_path_prefix}/full_major", gc_base.full_major)
post_endpoint("#{gc_path_prefix}/major", gc_base.major)
post_endpoint("#{gc_path_prefix}/minor", gc_base.minor)
post_endpoint("#{gc_path_prefix}/major_slice", gc_base.major_slice)
end |
Beta Was this translation helpful? Give feedback.
-
|
Can you try with the latest |
Beta Was this translation helpful? Give feedback.
-
|
Should I stick with ocaml:4.14.1, or would ocaml:5.1.1 be better?
|
Beta Was this translation helpful? Give feedback.
-
|
Those are great. OCaml Are you familiar with memtrace? This one has helped me a lot finding logical issues leading to memory leaks on the ocaml side. Here's an article about it: https://blog.janestreet.com/finding-memory-leaks-with-memtrace/ Typically, I run a couple of gc full major then look at all variable instantiated after the script started streaming and before the full major and still alive after the full major.. And.. booom: 
|
Beta Was this translation helpful? Give feedback.
-
|
I used the exact article to set up the viewer, but I still don't understand what I'm supposed to be looking at. It looks like everything is fine. Here's a new set of files: stats.tar.gz. Images
By the way, the memory graph looks a bit different. Dependencies
Build configVersion b573ac2 |
Beta Was this translation helpful? Give feedback.
-
|
Yeah it looks like all is good on the OCaml side. Could be a case of the GC being lazy. You said memory goes back to expected level after a full major? |
Beta Was this translation helpful? Give feedback.
-
I didn't say that, because I don't know what to expect. |
Beta Was this translation helpful? Give feedback.
-
Hi @toots, can you take a look at my
memtrace.ctfandstatistics.json?I am wondering if everything is working properly or if something is wrong.
stats.tar.gz
You can also find pictures of my plots here.
Images
The large drop in the
fragmentsgraph was caused by manualruntime.gc.full_majorandruntime.gc.compactcalls.Build config
Some settings
I turned off image decoding because the playlist is cluttered with different images.
Beta Was this translation helpful? Give feedback.
All reactions