Skip to content

Commit

Permalink
scheduler: tag executors with app id
Browse files Browse the repository at this point in the history
Thus far, we tagged executors with the (user, function) pair. This is
needed in multi-threaded environments, where different threads must
share the same executor. However, using only the (user, function) pair
prevents the execution concurrent applications with the same pair.

Instead, the correct thing to do is to track applications using the
(user, function) pair in conjunction with the application id.
  • Loading branch information
csegarragonz committed May 6, 2024
1 parent ddba013 commit c228c0f
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 6 deletions.
11 changes: 6 additions & 5 deletions src/scheduler/Scheduler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ int Scheduler::reapStaleExecutors()
long Scheduler::getFunctionExecutorCount(const faabric::Message& msg)
{
faabric::util::SharedLock lock(mx);
const std::string funcStr = faabric::util::funcToString(msg, false);
const std::string funcStr = faabric::util::funcToString(msg, true);
return executors[funcStr].size();
}

Expand All @@ -252,7 +252,8 @@ void Scheduler::executeBatch(std::shared_ptr<faabric::BatchExecuteRequest> req)
faabric::util::FullLock lock(mx);

bool isThreads = req->type() == faabric::BatchExecuteRequest::THREADS;
auto funcStr = faabric::util::funcToString(req);
auto funcStr = faabric::util::funcToString(req->messages(0), true);

int nMessages = req->messages_size();

// Records for tests - copy messages before execution to avoid races
Expand All @@ -265,8 +266,8 @@ void Scheduler::executeBatch(std::shared_ptr<faabric::BatchExecuteRequest> req)
// For threads we only need one executor, for anything else we want
// one Executor per function in flight.
if (isThreads) {
// Threads use the existing executor. We assume there's only
// one running at a time.
// Threads use the existing executor. There must only be one at a time,
// thus we use a function string that includes the app id
std::vector<std::shared_ptr<faabric::executor::Executor>>&
thisExecutors = executors[funcStr];

Expand Down Expand Up @@ -335,7 +336,7 @@ std::shared_ptr<faabric::executor::Executor> Scheduler::claimExecutor(
faabric::Message& msg,
faabric::util::FullLock& schedulerLock)
{
std::string funcStr = faabric::util::funcToString(msg, false);
std::string funcStr = faabric::util::funcToString(msg, true);

std::vector<std::shared_ptr<faabric::executor::Executor>>& thisExecutors =
executors[funcStr];
Expand Down
2 changes: 1 addition & 1 deletion src/util/func.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ std::string funcToString(const faabric::Message& msg, bool includeId)
std::string str = msg.user() + "/" + msg.function();

if (includeId) {
str += ":" + std::to_string(msg.id());
str += ":" + std::to_string(msg.appid());
}

return str;
Expand Down

0 comments on commit c228c0f

Please sign in to comment.