Python线程结束后join python多线程结束

转载

mob64ca140a1f7c 2023-08-11 14:44:23

文章标签 Python线程结束后join python 多线程 sed 死锁 文章分类 Python 后端开发

python多线程结束线程

This is the second and final part of the series. You can find the first part of the blog here. The first part covered basic python multi-threading. The second part discusses more advanced topics of multi-threading.

这是本系列的第二部分也是最后一部分。您可以在此处找到博客的第一部分。第一部分介绍了基本的python多线程。第二部分讨论多线程的更高级主题。

Python multi-threading is very similar to Dwight Schrute. If implemented properly it will be your good friend. If not, then we all know what happened in the snowball fight!

Python多线程与Dwight Schrute非常相似。如果实施正确，它将是您的好朋友。如果没有，那么我们都知道打雪仗发生了什么！

(DEADLOCK)

Deadlock occurs when thread is waiting for a resource. Let’s take an example of the episode “Murder” from the office when Michael, Andy, Dwight and Pam find themselves in a fake Mexican handoff.

当线程正在等待资源时发生死锁。让我们以迈克尔·安迪，德怀特和帕姆发现自己在假墨西哥移交中的办公室中的“ 谋杀案 ”为例。

Python线程结束后join python多线程结束_python

Let’s consider there are four guns (locks) and each person will need 2 guns (locks) to shoot the other person. They wait for 1000 sec before starting to shoot. Following code represents this

让我们考虑有四把枪(锁)，每个人需要两把枪(锁)才能射击另一个人。他们等待1000秒才能开始拍摄。以下代码代表了这一点

import threading
import timei_will_shoot_you = 1000gun_a = threading.Lock()
gun_b = threading.Lock()
gun_c = threading.Lock()
gun_d = threading.Lock()def mexican_standoff(gun1, gun2):
    global i_will_shoot_you
    name = threading.current_thread().getName()
    while i_will_shoot_you > 0:
        gun1.acquire()
        gun2.acquire()
        i_will_shoot_you -= 1
        print(f"{name}: Warning I will kill you after {i_will_shoot_you} sec")
        time.sleep(0.1)
        gun1.release()
        gun2.release()if __name__ == '__main__':
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_b), name='Michael').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_b, gun2=gun_c), name='Andy').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_c, gun2=gun_d), name='Pam').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_d, gun2=gun_a), name='Dwight').start()

Sample output

样品输出

Michael: Warning I will kill you after 999 sec
Pam: Warning I will kill you after 998 sec
Michael: Warning I will kill you after 997 sec
Pam: Warning I will kill you after 996 sec
Michael: Warning I will kill you after 995 sec
Pam: Warning I will kill you after 994 sec
Michael: Warning I will kill you after 993 sec
Pam: Warning I will kill you after 992 sec
Michael: Warning I will kill you after 991 sec
Pam: Warning I will kill you after 990 sec
Pam: Warning I will kill you after 989 sec
Michael: Warning I will kill you after 988 sec
Pam: Warning I will kill you after 987 sec
Michael: Warning I will kill you after 986 sec
Michael: Warning I will kill you after 985 sec
Dwight: Warning I will kill you after 984 sec
Dwight: Warning I will kill you after 983 sec
Pam: Warning I will kill you after 982 sec
Andy: Warning I will kill you after 981 sec
Andy: Warning I will kill you after 980 sec
Michael: Warning I will kill you after 979 sec
Dwight: Warning I will kill you after 978 sec
Dwight: Warning I will kill you after 977 sec

We have four locks, gun_a, gun_b, gun_c and gun_d. Each thread tries to acquire 2 locks (guns) before executing print statement (critical section). This may lead to a deadlock condition. Let’s say Pam has picked up gun_a (lock1) but is waiting to pick up gun_b (lock2). But turns out Andy has picked up gun_b (lock1) and is waiting for a different gun (lock2). This leads to a deadlock. As each character (thread) is waiting for a different lock to be available, while holding onto one gun (lock).

我们有四个锁，分别是gun_a，gun_b，gun_c和gun_d。每个线程在执行打印语句(关键部分)之前尝试获取2个锁(枪)。这可能导致死锁。假设Pam已拿起gun_a(lock1)，但正在等待拿起gun_b(lock2)。但是事实证明，安迪(Andy)拿起gun_b(lock1)，正在等待另一把枪(lock2)。这导致死锁。当每个角色(线程)在等待另一把锁可用时，同时握住一把枪(锁)。

One way to avoid this is, if threads try to acquire a common lock first. This way all the threads try to acquire one common lock first before trying to acquire a different lock. Thus, avoiding deadlock. We can edit the code so that all threads first try to acquire gun_a lock.

避免这种情况的一种方法是，如果线程首先尝试获取公共锁。这样，所有线程先尝试获取一个公共锁，然后再尝试获取其他锁。因此，避免了死锁。我们可以编辑代码，以便所有线程首先尝试获取gun_a锁。

threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_b), name='Michael').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_c), name='Andy').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_b), name='Pam').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_d), name='Dwight').start()

(DESERTED LOCK)

When there are multiple threads competing to acquire a lock to execute critical section and due to some error thread that has acquired the lock crashes before releasing the lock. Because of this other threads wait indefinitely to acquire the lock.

当有多个线程竞争获取锁以执行关键部分时，并且由于某些错误而导致获取锁的线程崩溃，因此在释放锁之前会崩溃。因此，其他线程会无限期等待以获取锁。

Let’s demonstrate this using the episode when Michael quits Dunder Mifflin and decides to moves to Colorado with Holly.

让我们用迈克尔退出邓德·米夫林(Dunder Mifflin)并决定与霍莉(Holly)搬到科罗拉多州时的一集来证明这一点。

import threadingleaving_show = threading.Lock()def leaving_the_show(character=None):
    # list of characters killed in the movie
    leaving_show.acquire()
    print(f"{character} has acquired the lock.")
    if character == "Michael":
        print(f"Sadly, {character} has decided to leave the show! :(")
        exit(1)
    leaving_show.release()
    print(f"{character} has released the lock.")if __name__ == '__main__':
    characters = ["Toby", "Dwight", "Angela",
                  "Jim", "Pam", "Michael", "Stanley", "Oscar", "Creed"]
    threads = list()# Defining different threads for each character
    for character in characters:
        threads.append(threading.Thread(
                                target=leaving_the_show,
                                kwargs=dict(character=character),
                                name=character)
                                )# This will start the threads
    for thread in threads:
        thread.start()# Waiting for all the threads to finish execution
    for thread in threads:
        thread.join()

Sample output

样品输出

Toby has acquired the lock.
Toby has released the lock.
Dwight has acquired the lock.
Dwight has released the lock.
Angela has acquired the lock.
Angela has released the lock.
Jim has acquired the lock.
Jim has released the lock.
Pam has acquired the lock.
Pam has released the lock.
Michael has acquired the lock.
Sadly, Michael has decided to leave the show! :(

In the code above, each of the thread acquire leaving_show lock to execute print (critical section) in function leaving_the_show. However, when thread Michael is executing it exits the program before releasing the lock. This leads to subsequent threads waiting infinitely to acquire the lock leaving_show.

在上面的代码中，每个线程都获取Leave_show锁，以在函数Leave_the_show中执行打印(关键部分)。但是，当线程Michael正在执行时，它在释放锁之前退出程序。这导致后续线程无限期地等待获取锁leaking_show。

To avoid this we can use try-finally for acquiring and releasing the lock.

为了避免这种情况，我们可以使用try-finally来获取和释放锁。

def leaving_the_show(character=None):
    # list of characters killed in the movie
    try:
        leaving_show.acquire()
        print(f"{character} has acquired the lock.")
        if character == "Dwight":
            print ("I, Dwight K Schrute will make sure that the show must go on!")
        if character == "Michael":
            print(f"Sadly, {character} had decided to leave the show! :(")
            exit(1)
    finally:
        leaving_show.release()
    print(f"{character} has released the lock.")

Sample output

样品输出

Toby has acquired the lock.
Toby has released the lock.
Dwight has acquired the lock.
I, Dwight K Schrute will make sure that the show must go on!
Dwight has released the lock.
Angela has acquired the lock.
Angela has released the lock.
Jim has acquired the lock.
Jim has released the lock.
Pam has acquired the lock.
Pam has released the lock.
Michael has acquired the lock.
Sadly, Michael had decided to leave the show! :(
Stanley has acquired the lock.
Stanley has released the lock.
Oscar has acquired the lock.
Oscar has released the lock.
Creed has acquired the lock.
Creed has released the lock.

(LIVELOCK)

Livelocks are similar to deadlocks. Livelocks occur when threads are designed to respond to actions of other threads (often to avoid deadlock), which leads to threads actively trying to resolve a conflict without making any progress.

活锁类似于死锁。当线程被设计为响应其他线程的动作(通常是为了避免死锁)时，就会发生动态锁，这导致线程主动尝试解决冲突而不取得任何进展。

To demonstrate livelock, let’s recall the Mexican standoff example that we used for deadlock. Each character will try to pick up gun1 first, when trying to pickup second gun it will check if gun2 is available to be picked up, if not gun1 will be politely dropped.

为了演示活锁，让我们回想一下用于死锁的墨西哥僵持示例。每个角色将首先尝试拾取gun1，在尝试拾取第二把枪时，它将检查gun2是否可以被拾取，如果不是，将礼貌地丢弃gun1。

import threadingi_will_shoot_you = 1000gun_a = threading.Lock()
gun_b = threading.Lock()
gun_c = threading.Lock()def mexican_standoff(gun1, gun2):
    global i_will_shoot_you
    name = name = threading.current_thread().getName()
    while i_will_shoot_you > 0:
        gun1.acquire()
        if not gun2.acquire(blocking=False):
            print(f"{name} politely dropped the gun to avoid deadlock")
            gun1.release()
        else:
            i_will_shoot_you -= 1
            print(f"Warning I will kill you after {i_will_shoot_you} sec")
            gun1.release()
            gun2.release()if __name__ == '__main__':
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_a, gun2=gun_b), name='Michael').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_b, gun2=gun_c), name='Andy').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_c, gun2=gun_b), name='Pam').start()
    threading.Thread(target=mexican_standoff, kwargs=dict(gun1=gun_b, gun2=gun_c), name='Dwight').start()

Sample output

样品输出

Warning I will kill you after 977 sec
Warning I will kill you after 976 sec
Warning I will kill you after 975 sec
Warning I will kill you after 974 sec
Pam politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Dwight politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Andy politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Andy politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Andy politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Andy politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Michael politely dropped the gun to avoid deadlock
Pam politely dropped the gun to avoid deadlock

As seen in the output of the code, threads try to acquire both the locks (gun1 and gun2), if gun2 is not available then gun1 lock is released.This leads to livelock situation.

从代码输出中可以看到，线程尝试同时获取两个锁(gun1和gun2)，如果gun2不可用，则gun1锁被释放，这导致了活锁情况。

Both deadlock and livelock cause program execution to not proceed. But there is a slight difference in behavior. In case of deadlocks threads are waiting for the resources, so CPU cycles are not consumed, while in livelock threads are actively trying to resolve the conflict leading to higher CPU usage. This is called ‘busy waiting’. CPU utilization is a good way to figure out if it’s a deadlock or a livelock.

死锁和活动锁都会导致程序无法继续执行。但是行为上略有不同。如果出现死锁，线程正在等待资源，因此不会占用CPU周期，而在活锁中，线程正在积极尝试解决导致更高CPU使用率的冲突。这称为“忙等待”。 CPU利用率是弄清楚是死锁还是活锁的好方法。

To avoid livelock, we can let the thread sleep for a random amount of time before trying to reacquire locks.

为避免发生活锁，我们可以让线程在尝试重新获取锁之前随机Hibernate一段时间。

Let’s modify the mexican_standoff function with the change.

让我们用更改来修改mexican_standoff函数。

def mexican_standoff(gun1, gun2):
    global i_will_shoot_you
    name = name = threading.current_thread().getName()
    while i_will_shoot_you > 0:
        gun1.acquire()
        if not gun2.acquire(blocking=False):
            print(f"{name} politely dropped the gun to avoid deadlock")
            gun1.release()
            time.sleep(random()/100)
        else:
            i_will_shoot_you -= 1
            print(f"Warning I will kill you after {i_will_shoot_you} sec")
            gun1.release()
            gun2.release()

In the function above threads sleep for a random time after dropping gun1 (releasing lock) and trying to reacquire the lock.

在上面的函数中，在释放gun1(释放锁)并尝试重新获取锁之后，线程会随机睡眠一段时间。

(SEMAPHORE)

Semaphore is much like mutex, it protects the critical section of the program, but it also provides additional functionalities. Semaphore allows multiple threads to access the critical section at a time. It keeps a counter to track number of threads accessing the critical section. As long as counter is more than zero threads can acquire the semaphore and decrement the counter. When counter hits zero, threads wait in a queue till semaphore is available again. Another difference between semaphore and mutex is, in case of semaphore, it can be acquired or released by different threads, while mutex can be acquired or released by same thread.

信号量与互斥量非常相似，它可以保护程序的关键部分，但还提供其他功能。信号量允许多个线程一次访问关键部分。它保留一个计数器来跟踪访问关键部分的线程数。只要计数器大于零，线程就可以获取信号量并使计数器递减。当计数器为零时，线程将在队列中等待，直到信号灯再次可用。信号量和互斥量之间的另一个区别是，在信号量的情况下，它可以由不同的线程获取或释放，而互斥量可以由同一线程获取或释放。

To demonstrate this let’s take example of Darryl leaving Dunder Mifflin and deciding to dance with his colleagues. In this case Darryl is the critical section and we will use semaphore to allow 2 threads (characters) to dance with Darryl (accessing critical section).

为了说明这一点，让我们以达里尔离开邓德·米夫林并决定与他的同事共舞为例。在这种情况下，Darryl是关键部分，我们将使用信号量允许2个线程(字符)与Darryl跳舞(访问关键部分)。

import threading
import timedancing = threading.Semaphore(2)def darryl_dancing():
    name = threading.current_thread().getName()
    with dancing:
        print(f"Darryl is dancing with {name}")
        time.sleep(1)if __name__ == '__main__':
    characters = ["Toby", "Creed", "Erin", "Kevin", "Nellie", "Oscar", "Clark"]
    threads = list()
    for character in characters:
        threads.append(threading.Thread(target=darryl_dancing, name=character))for thread in threads:
        thread.start()for thread in threads:
        thread.join()

Sample output

样品输出

Darryl is dancing with Toby
Darryl is dancing with Creed
Darryl is dancing with Erin
Darryl is dancing with Kevin
Darryl is dancing with Nellie
Darryl is dancing with Oscar
Darryl is dancing with Clark

In the code above we use dancing as semaphore object, which allows 2 threads to access the critical section at a time. While 2 threads are accessing the critical section, other threads will wait for current threads to release the semaphore. Also, if you notice I have used “with” keyword to acquire and release the semaphore in the code above. If we change semaphore definition to allow 1 thread at a time, then it becomes a binary semaphore and behaves just like mutex.

在上面的代码中，我们将舞动用作信号量对象，该对象允许2个线程一次访问关键部分。当2个线程正在访问关键部分时，其他线程将等待当前线程释放信号量。另外，如果您注意到我在上面的代码中使用了“ with”关键字来获取和释放信号量。如果我们将信号量定义更改为一次允许1个线程，那么它将变为二进制信号量，其行为类似于互斥锁。

(RACE CONDITION)

Let’s talk about real devil in multi-threading. When multiple threads execute critical section, due to scheduling of threads by the operating system it may result in different results.

让我们谈谈多线程中的真正魔鬼。当多个线程执行关键部分时，由于操作系统对线程的调度，可能会导致不同的结果。

Let’s take 2 functions kevin_multiplication which will multiply accounts variable by 3 and oscar_addition which will add 10 to the accounts variable. Function “calculating” simulates threads spending time in calculations. As expected, thread Kevin takes 5 units to complete calculation and thread Oscar take 3 units.

让我们使用2个函数kevin_multiplication，它将accounts变量乘以3，将oscar_addition乘以10的accounts变量。函数“计算”模拟线程在计算中花费的时间。正如预期的那样，线程Kevin需要5个单位才能完成计算，线程Oscar需要3个单位。

import threadingaccounts = 1
doomsday = threading.Lock()def calculating(checks):
    task = 0
    for i in range(checks * 100000):
        task += 1def kevin_multiplication():
    global accounts
    calculating(5)
    with doomsday:
        print("Kevin is editing the accounts")
        accounts *= 3def oscar_addition():
    global accounts
    calculating(3)
    with doomsday:
        print("Oscar is editing the accounts")
        accounts += 10if __name__ == '__main__':
    threads = list()
    for i in range(10):
        threads.append(threading.Thread(target=kevin_multiplication, name="Kevin"))
        threads.append(threading.Thread(target=oscar_addition, name="Oscar"))for thread in threads:
        thread.start()for thread in threads:
        thread.join()print(f"Final account tally is: {accounts}")

Sample output

样品输出

First run:
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Final account tally is: 926689Second run:
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Final account tally is: 399709

In the sample output above, in first run final account tally was 926689 while in the second run tally was 399709. This is due to the scheduling of the threads. If Dwight had setup doomsday device it would have beeped at least once already!

在上面的示例输出中，第一次运行时最终帐户的理算结果为926689，而第二次运行时最终的理算结果为399709。这是由于线程的调度所致。如果德怀特(Dwight)设置了世界末日设备，它将至少已经发出一次哔声！

To fix this we have to change how threads are modifying accounts variable regardless of the scheduling of the threads. We can use barrier in python threading module for this. Thread Kevin will modify the variable first and when it has modified the variable it will notify thread Oscar to start updating the variable. This way we will always get a consistent value.

为了解决这个问题，我们必须更改线程修改帐户变量的方式，而与线程的调度无关。我们可以在python线程模块中使用barrier 。线程Kevin将首先修改变量，并在修改变量后通知线程Oscar开始更新变量。这样，我们将始终获得一致的价值。

import threadingaccounts = 1
doomsday = threading.Lock()
beat_doomsday_device = threading.Barrier(20)def calculating(checks):
    task = 0
    for i in range(checks * 100000):
        task += 1def kevin_multiplication():
    global accounts
    calculating(5)
    with doomsday:
        print("Kevin is editing the accounts")
        accounts *= 3
    beat_doomsday_device.wait()def oscar_addition():
    global accounts
    calculating(3)
    beat_doomsday_device.wait()
    with doomsday:
        print("Oscar is editing the accounts")
        accounts += 10if __name__ == '__main__':
    threads = list()
    for i in range(10):
        threads.append(threading.Thread(target=kevin_multiplication, name="Kevin"))
        threads.append(threading.Thread(target=oscar_addition, name="Oscar"))for thread in threads:
        thread.start()for thread in threads:
        thread.join()print(f"Final account tally is: {accounts}")

Sample output

样品输出

First run:
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Final account tally is: 59149Second run:
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Kevin is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Oscar is editing the accounts
Final account tally is: 59149

In the code sample above, we have created a barrier beat_doomsday_device. Thread Kevin complete execution and waits at barrier. While thread Oscar waits for thread Kevin to finish execution before changing value of variable accounts.

在上面的代码示例中，我们创建了一个屏障beat_doomsday_device。线程Kevin完成执行并等待障碍。在线程Oscar等待线程Kevin完成执行之前，更改变量帐户的值。

This brings us to the end of the blog! This was a great learning experience for me and I hope it was useful to you too. I leave you with my favorite the Office quote.

这将我们带到博客的结尾！这对我来说是一次很棒的学习经历，我希望它对您也有用。我给我留下我最喜欢的Office报价。

Python线程结束后join python多线程结束_python_02