Merge branch 'master' into g88_trying_to_make_keepalive_local

.github/workflows/tests.yml

@@ -0,0 +1,26 @@
+name: Tests
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test-normal-compile:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Install Valgrind
+        run: sudo apt-get install -y valgrind
+      - name: Run tests with standard compilation flags
+        working-directory: tests/
+        run: ./normal_compile.sh
+
+  test-optimized-compile:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Install Valgrind
+        run: sudo apt-get install -y valgrind
+      - name: Run tests with optimized compilation flags
+        working-directory: tests/
+        run: ./optimized_compile.sh

README.md

@@ -1,4 +1,5 @@
-![Build status](http://178.62.170.124:3000/pithikos/c-thread-pool/badge/?branch=master)
+[![GitHub Actions](https://github.com/Pithikos/C-Thread-Pool/workflows/tests.yml/badge.svg?branch=master)](https://github.com/Pithikos/C-Thread-Pool/actions?query=workflow%3Atests+branch%3Amaster)
+
 
 # C Thread Pool
 
@@ -65,3 +66,5 @@ You are very welcome to contribute. If you have a new feature in mind, you can a
 * For coding style simply try to stick to the conventions you find in the existing codebase.
 * Tests: A new fix or feature should be covered by tests. If the existing tests are not sufficient, we expect an according test to follow with the pull request.
 * Documentation: for a new feature please add documentation. For an API change the documentation has to be thorough and super easy to understand.
+
+If you wish to **get access as a collaborator** feel free to mention it in the issue https://github.com/Pithikos/C-Thread-Pool/issues/78

docs/Design.md

@@ -1,47 +1,47 @@
 ## High level
-	
+
 	Description: Library providing a threading pool where you can add work on the fly. The number
 	             of threads in the pool is adjustable when creating the pool. In most cases
 	             this should equal the number of threads supported by your cpu.
-	         
+
 	             For an example on how to use the threadpool, check the main.c file or just read
 	             the documentation found in the README.md file.
-	
+
 	             In this header file a detailed overview of the functions and the threadpool's logical
-	             scheme is presented in case you wish to tweak or alter something. 
-	
-	
-	
-	              _______________________________________________________        
+	             scheme is presented in case you wish to tweak or alter something.
+
+
+
+	             _______________________________________________________
 	            /                                                       \
-	            |   JOB QUEUE        | job1 | job2 | job3 | job4 | ..   |
+	            |   JOB QUEUE       | job1 | job2 | job3 | job4 | ..    |
 	            |                                                       |
 	            |   threadpool      | thread1 | thread2 | ..            |
 	            \_______________________________________________________/
-	
-	
+
+
 	   Description:       Jobs are added to the job queue. Once a thread in the pool
-	                      is idle, it is assigned with the first job from the queue(and
-	                      erased from the queue). It's each thread's job to read from 
-	                      the queue serially(using lock) and executing each job
+	                      is idle, it is assigned the first job from the queue (and that job is
+	                      erased from the queue). It is each thread's job to read from
+	                      the queue serially (using lock) and executing each job
 	                      until the queue is empty.
-	
-	
+
+
 	   Scheme:
-	
-	   thpool______                jobqueue____                      ______ 
+
+	   thpool______                jobqueue____                      ______
 	   |           |               |           |       .----------->|_job0_| Newly added job
 	   |           |               |  rear  ----------'             |_job1_|
 	   | jobqueue----------------->|           |                    |_job2_|
-	   |           |               |  front ----------.             |__..__| 
+	   |           |               |  front ----------.             |__..__|
 	   |___________|               |___________|       '----------->|_jobn_| Job for thread to take
-	
-	
-	   job0________ 
+
+
+	   job0________
 	   |           |
 	   | function---->
 	   |           |
 	   |   arg------->
-	   |           |         job1________ 
+	   |           |         job1________
 	   |  next-------------->|           |
 	   |___________|         |           |..

docs/FAQ.md

@@ -1,35 +1,36 @@
+### Why isn't `pthread_exit()` used to exit a thread?
 
-###Why isn't pthread_exit() used to exit a thread?
-`thread_do` used to use pthread_exit(). However that resulted in
-hard times of testing for memory leaks. The reason is that on pthread_exit()
+`thread_do` used to use `pthread_exit()`. However that resulted in
+hard times of testing for memory leaks. The reason is that on `pthread_exit()`
 not all memory is freed bt pthread (probably for future threads or false
 belief that the application is terminating). For these reasons a simple return
 is used.
 
 Interestingly using `pthread_exit()` results in much more memory being allocated.
 
 
-###Why do you use sleep() after calling thpool_destroy()?
-This is needed only in the tests. The reason is that if you call thpool_destroy
-and then exit immedietely, maybe the program will exit before all the threads
+### Why do you use `sleep()` after calling `thpool_destroy()`?
+
+This is needed only in the tests. The reason is that if you call `thpool_destroy()`
+and then exit immediately, maybe the program will exit before all the threads
 had the time to deallocate. In that way it is impossible to check for memory
 leaks.
 
-In production you don't have to worry about this since if you call exit,
-immedietely after you destroyied the pool, the threads will be freed
-anyway by the OS. If you eitherway destroy the pool in the middle of your
+In production you don't have to worry about this since if you call `exit()`,
+immediately after you destroyed the pool, the threads will be freed
+anyway by the OS. If you anyway destroy the pool in the middle of your
 program it doesn't matter again since the program will not exit immediately
 and thus threads will have more than enough time to terminate.
 
 
+### Why does `wait()` use all my CPU?
 
-###Why does wait() use all my CPU?
-Notice: As of 11-Dec-2015 wait() doesn't use polling anymore. Instead a conditional variable is being used so in theory there should not be any CPU overhead.
+Notice: As of 11-Dec-2015 `wait()` doesn't use polling anymore. Instead a conditional variable is being used so in theory there should not be any CPU overhead.
 
 Normally `wait()` will spike CPU usage to full when called. This is normal as long as it doesn't last for more than 1 second. The reason this happens is that `wait()` goes through various phases of polling (what is called smart polling).
 
  * Initially there is no interval between polling and hence the 100% use of your CPU.
  * After that the polling interval grows exponentially.
  * Finally after x seconds, if there is still work, polling falls back to a very big interval.
- 
+
 The reason `wait()` works in this way, is that the function is mostly used when someone wants to wait for some calculation to finish. So if the calculation is assumed to take a long time then we don't want to poll too often. Still we want to poll fast in case the calculation is a simple one. To solve these two problems, this seemingly awkward behaviour is present.

example.c

@@ -13,16 +13,11 @@
 
 #include <stdio.h>
 #include <pthread.h>
+#include <stdint.h>
 #include "thpool.h"
 
-
-void task1(){
-	printf("Thread #%u working on task1\n", (int)pthread_self());
-}
-
-
-void task2(){
-	printf("Thread #%u working on task2\n", (int)pthread_self());
+void task(void *arg){
+	printf("Thread #%u working on %d\n", (int)pthread_self(), (int) arg);
 }
 
 
@@ -33,11 +28,11 @@ int main(){
 
 	puts("Adding 40 tasks to threadpool");
 	int i;
-	for (i=0; i<20; i++){
-		thpool_add_work(thpool, (void*)task1, NULL);
-		thpool_add_work(thpool, (void*)task2, NULL);
+	for (i=0; i<40; i++){
+		thpool_add_work(thpool, task, (void*)(uintptr_t)i);
 	};
 
+	thpool_wait(thpool);
 	puts("Killing threadpool");
 	thpool_destroy(thpool);
 

tests/memleaks.sh

@@ -28,25 +28,48 @@ function test_thread_free { #threads
 }
 
 
+function _test_thread_free_multi { #threads
+	output=$(valgrind --leak-check=full --track-origins=yes ./test "$1" 2>&1 > /dev/null)
+	heap_usage=$(echo "$output" | grep "total heap usage")
+	allocs=$(extract_num "[0-9]* allocs" "$heap_usage")
+	frees=$(extract_num "[0-9]* frees" "$heap_usage")
+	if (( "$allocs" == 0 )); then
+		err "Allocated 0 times. Something is wrong.." "$output"
+		return 1
+	fi
+	if (( "$allocs" != "$frees" )); then
+		err "Allocated $allocs times but freed only $frees" "$output"
+		return 1
+	fi
+	#echo "Allocs: $allocs    Frees: $frees"
+}
+
+
 # This is the same with test_many_thread_allocs but multiplied
-function test_thread_free_multi { #threads #times
+function test_thread_free_multi { #threads #times #nparallel
 	echo "Testing multiple threads creation and destruction in pool(threads=$1 times=$2)"
 	compile src/no_work.c
-	for ((i = 1; i <= $2; i++)); do
+	pids=()
+	nparallel="${3:-10}"
+
+	# Run tests in p
+	for (( i = 1; i <= "$2"; i++ )); do
+
+		# Run test in background
 		python -c "import sys; sys.stdout.write('$i/$2\r')"
-		output=$(valgrind --leak-check=full --track-origins=yes ./test "$1" 2>&1 > /dev/null)
-		heap_usage=$(echo "$output" | grep "total heap usage")
-		allocs=$(extract_num "[0-9]* allocs" "$heap_usage")
-		frees=$(extract_num "[0-9]* frees" "$heap_usage")
-		if (( "$allocs" == 0 )); then
-			err "Allocated 0 times. Something is wrong.." "$output"
-			exit 1
-		fi
-		if (( "$allocs" != "$frees" )); then
-			err "Allocated $allocs times but freed only $frees" "$output"
-			exit 1
+		_test_thread_free_multi "$1" &
+		pids+=($!)
+
+		# Wait for 10 background jobs to finish
+		if (( "$i" % 10 == 0 )); then
+			for pid in ${pids[@]}; do
+				wait $pid
+				if (( $? != 0 )); then
+					err "Test failed" "Test failed"
+				fi
+			done
+			pids=()
 		fi
-		#echo "Allocs: $allocs    Frees: $frees"
 	done
 	echo
 }
@@ -62,7 +85,12 @@ test_thread_free 8
 test_thread_free 1
 test_thread_free 20
 test_thread_free_multi 4 20
-test_thread_free_multi 3 1000
-test_thread_free_multi 100 100
+
+# test_thread_free_multi 3 1000  # Takes way too long
+test_thread_free_multi 3 200
+
+# test_thread_free_multi 100 100  # Takes way too long
+test_thread_free_multi 100 20 1
+
 
 echo "No memory leaks"

tests/src/conc_increment.c

@@ -20,7 +20,7 @@ int main(int argc, char *argv[]){
 
 	char* p;
 	if (argc != 3){
-		puts("This testfile needs excactly two arguments");
+		puts("This testfile needs exactly two arguments");
 		exit(1);
 	}
 	int num_jobs    = strtol(argv[1], &p, 10);

tests/src/pause_resume.c

@@ -24,7 +24,7 @@ int main(int argc, char *argv[]){
 
 	char* p;
 	if (argc != 2){
-		puts("This testfile needs excactly one arguments");
+		puts("This testfile needs exactly one arguments");
 		exit(1);
 	}
 	int num_threads = strtol(argv[1], &p, 10);