shtSrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSKJrJrJrJr SSKJrJrJrJrJrJr SSKrSSKJr SSKJ r \S- r!SSK"J#r# \#"5 SS K$J%r% S r&SS K$J(r( S r)SSK*r+S r,\ RZ"\)S 5r.\ RZ"\&S5r/\)=(a \(R`S:Hr1\ RZ"\1S5r2\RfRiS5r5\RfRiS5r6\RnS:*r8\8r9\5(+r:Sr;\,(aSr"SS\?5r@"SS\@5rA"SS\?5rB"SS \@5rC"S!S"\@5rD"S#S$\@5rES%rFS&rG\ RrI"S'S(\?5rJS)rK\G"\I\R5rM\G"\K"S*56rN\5(d\G"\K"S+56rO\G"\K"S,56rP\G"\K"S-56rQ"S.S/\5rR\ RS0;rT\ RZ"\TS15"S2S3\R55rU\UR5 "S4S5\?5rW"S6S7\W\R5rX\XR5 "S8S9\5rY\"S:S;\Y55rZ\ZR5 \"S<S=\Y55r[\"S>S?\Y55r\\\2"S@SA\Y555r]\\/"SBSC\Y555r^\"SDSE\55r_\\."SFSG\555r`\aSH:Xa\ R"5 gg!\'a S r&GNf=f!\'a S r)GNf=f!\'a S r,GNf=f)Iz Tests the parallel backend N)jit vectorize guvectorizeset_num_threads)temp_directoryoverride_configTestCasetagskip_parfors_unsupported linux_only)_TIMEOUT)configgN@)_check_tbb_version_compatible)tbbpoolTF)omppoolzOpenMP threadpool requiredzTBB threadpool requiredGNUzGNU OpenMP only testswindarwinlc4[R"U5U-$N)nponesnvs u/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/numba/tests/test_parallel_backend.pyfoorHs 771:>c[R"[R"X45[R"X455nU[R"U5-U-$r)rdotrarange)rrxs rlinalgr#Ms> FF277A6?BGGQFO 4299Q<!##rc4[R"U5U-$r)rr!rs rr#r#Qsyy|arc X-$r)abs r ufunc_foor)Vs 5LrcX-US'g)Nrr&)r'r(outs r gufunc_foor,Zs UCFrc\rSrSrSrSrg)runnable^c Xlgr_options)selfoptionss r__init__runnable.__init___s rr1N)__name__ __module__ __qualname____firstlineno__r5__static_attributes__r&rrr.r.^s rr.c\rSrSrSrSrg) jit_runnercc[S0URD6"[5nSnSn[X#5nU"X#5n[RR XE5 gN r&)rr2rrtestingassert_allcloser3cfuncr'r(expectedgots r__call__jit_runner.__call__esF$dmm$S)  q9Ak ""81rr&Nr7r8r9r:rIr;r&rrr=r=c2rr=c \rSrSrSrSrSrg) mask_runnernc XlX lgr)runnermask)r3rQrRr4s rr5mask_runner.__init__os   rcpUR(a[UR5 UR5 gr)rRrrQ)r3s rrImask_runner.__call__ss 99 DII & r)rRrQNr7r8r9r:r5rIr;r&rrrNrNns rrNc\rSrSrSrSrg) linalg_runner{c[S0URD6"[5nSnSn[X#5nU"X#5n[RR XE5 gr@)rr2r#rrCrDrEs rrIlinalg_runner.__call__}sF$dmm$V,  !<Ak ""81rr&NrKr&rrrXrX{rLrrXc\rSrSrSrSrg)vectorize_runnerc$[S/40URD6"[5n[RR S5R [R 5=p#[X#5nU"X#5n[RRXE5 g)Nz(f4, f4)rB) rr2r)rrandomastypefloat32rCrDrEs rrIvectorize_runner.__call__sf:,8$--8C   $++BJJ77Q?Ak ""81rr&NrKr&rrr]r]s2rr]c\rSrSrSrSrg)guvectorize_runnerc*S/n[US40URD6"[5n[RR S5R [R 5=p4[X45nU"X45n[RRXV5 g)Nz(f4, f4, f4[:])z (),()->()rB) rr2r,rr`rarbr)rCrD)r3sigrFr'r(rGrHs rrIguvectorize_runner.__call__sn !C> >zJ   $++BJJ77Q?Ak ""81rr&NrKr&rrrererLrrec *URS5n[R"5 [[ [ U5S-55H n[ R"U5nU"5 M" g![anURU5 SnAgSnAff=f)Nqueueg?) get faulthandlerenablerangeintlenr`choice Exceptionput)fnlistkwargsq_fnes rchooserr{sp 7As3v;,-.Av&B D/  asAA-- B7B  Bc^^UU4SjnU$)Nc >T "5nSU0n[S5Vs/sHnT "[U4US9PM nnUHnUR5 M UHnUR5 M UR 5(d/nUR 5(d7UR UR S55 UR 5(dM7Sn[USRUVs/sHn[U5PM sn5-5egs snfs snf)NrkrA)targetargsrvFz)Error(s) occurred in delegated runner: %s ) ror{startjoinemptyappendrl RuntimeErrorrepr) rurwkwsithstherrors_msgr"parallel_class queue_impls r run_compile$compile_factory..run_compiles Lla" 1WF9SI  "B HHJB GGIwwyyFggii aeeEl+ggii?Dtdii&0I&Qa&0I&JJK K  "1Js DDr&)rrrs`` rcompile_factoryrsL rc \rSrSrSrSrSrg)_proc_class_implcXlgr_method)r3methods rr5_proc_class_impl.__init__s rcf[R"UR5nUR"U0UD6$r)multiprocessing get_contextrProcess)r3rrvctxs rrI_proc_class_impl.__call__s*))$,,7{{D+F++rrNrVr&rrrrs ,rrcrUS:XaSn[R"U5n[U5nURnX#4$)Ndefault)rrrQueue)rrprocrks r_get_mp_classesrs:   % %f -C F #D IIE ;rspawnfork forkserverrcV\rSrSrSr\"SS9\"SSS9\"SSS9\"SS9\"SSS9\"SS9\"SSS9\"SSSS 9\"SS9\"SSS9\"SSSS 9/ r \ (d-\"SSS 9\"SSSS 9\"SSS 9\"SSSS 9/r \ R\ 5 \ RS :a/rOS S /r/r\ H$r\Hr\R'\"\\55 M M& SS/r\R'S5 \(a"\R'S5 \R'S5 \"S\ (+S 9/\"SSS9/\"SSS9/\ \S.rSS1rSSjrSrg)TestParallelBackendBasez. Base class for testing the parallel backends T)nopython)rcache)rnogilparallel)rr~)rr~r)rr)rrr threadingr`multiprocessing_spawnmultiprocessing_forkmultiprocessing_forkserver)concurrent_jitconcurrent_vectorizeconcurrent_guvectorizeconcurrent_mix_useconcurrent_mix_use_masksomptbbcZ[URR5Ul[ SUR5 US:Xa [ U5 OUS:Xa [ U5 OUS:Xa [U5 OUS:Xa [U5 OUS:Xa [U5 O}US:Xai[ [/n[(a*UR[ 5 UR[5 [R"U5 UH nU"U5 M O[SU-5eSSS5 g!,(df  g=f) N CACHE_DIRrrrrmultiprocessing_defaultr`zUnknown parallelism supplied %s)r __class__r7 _cache_dirr thread_implfork_proc_implforkserver_proc_implspawn_proc_impldefault_proc_impl _HAVE_OS_FORKrr`shuffle ValueError)r3ru parallelismpsimpls rr#TestParallelBackendBase.run_compiles()@)@A [$// :k)F# 66v& <<$V, 77' 99!&)(!?3 =IIn-II23r"DL!5 CEE+; : :s CD D*)rN)r)r7r8r9r:__doc__r=rXr]re all_impls_parfors_unsupported parfor_implsextendrNUMBA_NUM_THREADSmasks mask_implsrrRrrNrrrunners safe_backendsrr;r&rrrrs D!D-D-t$t40$'$z:$zFD)D<D4H I t 4 t4 @ 4$ 7 4$d C  & !#AJD   k$5 6)K./1278 4H0H J  d: >!  Z @# ($. GENMErr)rr workqueuezThreading layer not explicitc(\rSrSrSr\S5rSrg)TestParallelBackendi6a]These are like the numba.tests.test_threadsafety tests but designed instead to torture the parallel backend. If a suitable backend is supplied via NUMBA_THREADING_LAYER these tests can be run directly. This test class cannot be run using the multiprocessing option to the test runner (i.e. `./runtests -m`) as daemon processes cannot have children. cURHNnURR5H-up#SU-S-U-nSnU"X15nXFl[ XU5 M/ MP g)Ntest_rxc^^UU4SjnU$)Nc>[R"5nUR(aSnURU5 gUR TTS9 g)Nz)daemonized processes cannot have children)r)rcurrent_processdaemonskipTestr)r3selfprocrrps r test_methodBTestParallelBackend.generate..methgen..test_methodJs<#2#B#B#D#??#ND MM$/ ,,Tq,Arr&)rrrs`` rmethgen-TestParallelBackend.generate..methgenIsB'&r)rritemsr7setattr)clsrnamermethnamerrys rgenerateTestParallelBackend.generateCs\A!kk//1 "Q;,t3 'T%& r*2!rr&N)r7r8r9r:r classmethodrr;r&rrrr6s++rrcN\rSrSr\\\R"SS5S.rSr Sr Sr g) TestInSubprocessi[Frrrc[R"U[R[RUS9n[R"[ UR 5nUR5 UR5upVURS:wa,[SUR<SUR5<S35eUR5UR54UR5 $!UR5 f=f)Nstdoutstderrenvrprocess failed with code : stderr follows r subprocessPopenPIPErTimer _TEST_TIMEOUTkillr communicate returncodeAssertionErrordecodecancelr3cmdlinerpopentimeoutr+errs rrun_cmdTestInSubprocess.run_cmd`s  (2(2%(* //-<  MMO((*HC1$$%%szz|566::<- NN GNN s A=C((C:c[RR5n[U5US'[R SSU/nUR XC5$)NNUMBA_THREADING_LAYERz-mznumba.runtests)osenvironcopystrsys executabler )r3testthreading_layerenv_copyrs rrun_test_in_separate_process-TestInSubprocess.run_test_in_separate_processssD::??$,/,@()>>4)94@||G..rr&N) r7r8r9r: skip_no_tbb skip_no_ompunittestskipIfbackendsr rr;r&rrrr[s'""%__UB79H&/rrc<\rSrSrSrSr\S5r\S5rSr g)TestSpecificBackendiza This is quite contrived, for each test in the TestParallelBackend tests it generates a test that will run the TestParallelBackend test in a new python process with an environment modified to ensure a specific threadsafe backend is used. This is with view of testing the backends independently and in an isolated manner such that if they hang/crash/have issues, it doesn't kill the test suite. Fc ^^ URn[RnSU-S-U-nU<SU<SU<3m UU 4SjnSU<SU<ST<3n [X [ S5"U"U555 g)Nrrx.c`>URTT5upUR(a[SU<SU<S35 [R"SU5nUb UR UR S55 URSU5 URSU;5 URSU;5 g) Nz stdout: "z " stderr: ""z\.\.\. skipped '(.*?)'rOKFAILERROR) r_DEBUGprintresearchrgroupassertIn assertTrue)r3orzmbackendinjected_methods r test_template2TestSpecificBackend._inject..test_templates44_gNDA{{1a@A 3Q7A} aggaj) MM$ " OOF!O , OOG1, -r long_running)r8rr7rr ) rrrr1 backend_guardthemodtheclsrr3 injected_testr2s ` @r_injectTestSpecificBackend._injectsf$--Q;$t+(.A .,-dG< N#M-$@A Crc VURR5HupURHvnURR 5HUnUS;a,US:Xa&[ R RS5(aM5US;aUS:XaMCURX4X5 MW Mx M g)N)rr`rlinux)rr`r) rrrrkeysrplatform startswithr:)rr1r6rrs rrTestSpecificBackend.generates&)ll&8&8&: "G__KK,,.D??5(LL33G<< 44#{2 KK@/%';rr&N) r7r8r9r:rr(rr:rr;r&rrr r zs8FCC.AArr cl\rSrSrSrS\R R\5-r SS\ 0-r S Sjr Sr g) ThreadLayerTestHelperizH Helper class for running an isolated piece of code based on a template z%raif 1: import sys sys.path.insert(0, "%(here)r") import multiprocessing import numpy as np from numba import njit import numba try: import threading_backend_usecases except ImportError as e: print("DEBUG:", sys.path) raise e import os sigterm_handler = threading_backend_usecases.sigterm_handler busy_func = threading_backend_usecases.busy_func def the_test(): %%s if __name__ == "__main__": the_test() hereNcZUc,[RR5n[S5US'[R "U[R [R US9n[R"[UR5nUR5 UR5upVURS:wa,[SUR<SUR5<S35eUR!5 UR5UR54$!UR!5 f=f)Nrrrrrrr)rrrrrrrrrrrrrrrrrrs rr ThreadLayerTestHelper.run_cmds ;**//#C+.u:C' (  (2(2%(* //-<  MMO((*HC1$$%%szz|566% NN zz|SZZ\)) NN s ADD*r&r) r7r8r9r:rrpathdirname__file___heretemplater r;r&rrrCrCs: 277??8, ,E, 5/-H0*rrCcj\rSrSrSrSr\\\R"SS5S.r \ S5r \ S5r Srg ) TestThreadingLayerSelectioniz8 Checks that numba.threading_layer() reports correctly. Frrc ^^U4SjnST-n[X[S5"U"U555 g)Nc>SnURUT--n[RSU/n[RR 5n[ T5US'URX4S9upVUR(a [XV5 gg)Nzif 1: X = np.arange(1000000.) Y = np.arange(1000000.) Z = busy_func(X, Y) assert numba.threading_layer() == '%s' -crr) rKrrrrrrr r(r))r3bodyrunmerrr+r r1s rr3:TestThreadingLayerSelection._inject..test_templatesrD MMTG^4E~~tU3G**//#C+.w @rclURR5HupURX5 M gr)rrr:)rr1r6s rr$TestThreadingLayerSelection.generate s(&)ll&8&8&: "G KK /';rr&N)r7r8r9r:rr(rrrrrrr:rr;r&rrrMrMsSF""%__UB79H@@(00rrMc|\rSrSrS\4Sjr\\S55r\\S55r \S5r \S5r Sr S r g ) TestThreadingLayerPriorityienv_varc[RR5nSUS'XS'SUS3n[RS[ R "U5/nURXBS9 g) zJTest setting priority via env var NUMBA_THREADING_LAYER_PRIORITY. rrNUMBA_THREADING_LAYER_PRIORITYa import numba # trigger threading layer decision # hence catching invalid THREADING_LAYER_PRIORITY @numba.jit( 'float64[::1](float64[::1], float64[::1])', nopython=True, parallel=True, ) def plus(x, y): return x + y captured_envvar = list("a ".split()) assert numba.config.THREADING_LAYER_PRIORITY == captured_envvar, "priority mismatch" assert numba.threading_layer() == captured_envvar[0], "selected backend mismatch" rPrQN)rrrrrtextwrapdedentr )r3r[rcodecmds r each_env_var'TestThreadingLayerPriority.each_env_varsojjoo'0 #$07 ,- )*1 2( NN  OOD !  S "rc/SQn[R"U5H%nSRU5nURU5 M' g)Nr ) itertools permutationsrrb)r3rrr[s rtest_valid_env_var-TestThreadingLayerPriority.test_valid_env_var8s9.''0AhhqkG   g &1rcSnUR[5nURU5 SSS5 SH)nURU[ WR 55 M+ g!,(df  N>=f)Nztbb omp workqueue notvalidhere)z!THREADING_LAYER_PRIORITY invalid:zIt must be a permutation of) assertRaisesrrbr-r exception)r3r[raisesmsgs rtest_invalid_env_var/TestThreadingLayerPriority.test_invalid_env_var@s^3   ~ .&   g &/ C MMSEC(8(8$9 :  / .s A"" A0c8SHnURU5 M g)N)zomp tbb workqueuezomp workqueue tbbrbr3r[s rtest_omp#TestThreadingLayerPriority.test_ompLAG   g &Brc8SHnURU5 M g)N)ztbb omp workqueueztbb workqueue omprrrss rtest_tbb#TestThreadingLayerPriority.test_tbbQrvrc8SHnURU5 M g)N)zworkqueue tbb ompzworkqueue omp tbbrrrss rtest_workqueue)TestThreadingLayerPriority.test_workqueueVsAG   g &Brr&N)r7r8r9r:rrbrrrhrortrxr{r;r&rrrZrZst#C#B'' ;;'''''rrZct\rSrSrSrSr\S5r\S5r Sr \ R"\ S5S5rS rg ) TestMiscBackendIssuesi[zD Checks fixes for the issues with threading backends implementation FcSn[RSU/n[RR 5nSUS'SUS'UR X#S9 g) z( Tests that OMP does not overflow stack aif 1: from numba import vectorize, threading_layer import numpy as np @vectorize(['f4(f4,f4,f4,f4,f4,f4,f4,f4)'], target='parallel') def foo(a, b, c, d, e, f, g, h): return a+b+c+d+e+f+g+h x = np.ones(2**20, np.float32) foo(*([x]*8)) assert threading_layer() == "omp", "omp not found" rPrr100K OMP_STACKSIZErQNrrrrrr r3rSrrs rtest_omp_stack_overflow-TestMiscBackendIssues.test_omp_stack_overflowbsN  >>4/jjoo', #$%O W &rcSn[RSU/n[RR 5nSUS'SUS'UR X#S9 g) zY Tests that TBB works well with single thread https://github.com/numba/numba/issues/3440 aMif 1: from numba import njit, prange, threading_layer @njit(parallel=True) def foo(n): acc = 0 for i in prange(n): acc += i return acc foo(100) assert threading_layer() == "tbb", "tbb not found" rPrr1rrQNrrs rtest_single_thread_tbb,TestMiscBackendIssues.test_single_thread_tbbysO  >>4/jjoo', #$#&   W &rcjSn[RSU/n[RR 5nSUS'SUS'UR X#S9upEg ![ aWnUR(a [WW5 [U5nURSU5 S nURX5 S nAg S nAff=f) zG Tests workqueue raises sigabrt if a nested parallel call is performed aif 1: from numba import njit, prange import numpy as np @njit(parallel=True) def nested(x): for i in prange(len(x)): x[i] += 1 @njit(parallel=True) def main(): Z = np.zeros((5, 10)) for i in prange(Z.shape[0]): nested(Z[i]) return Z main() rPrr4rrQzfailed with codezTNumba workqueue threading layer is terminating: Concurrent access has been detected.N) rrrrrr rr(r)rr-) r3rSrrr+r rze_msgrGs r+test_workqueue_aborts_on_nested_parallelismATestMiscBackendIssues.test_workqueue_aborts_on_nested_parallelisms &>>4/jjoo'2 #$#&   +||G|5HC +{{c3FE MM,e 4?H MM( * * +sA B2A B--B2zTest needs fork(2)cSn[RSU/n[RR 5nSUS'SUS'UR X#S9 g)Naif 1: from numba import njit, prange, threading_layer import numpy as np import multiprocessing if __name__ == "__main__": # Need for force fork context (OSX default is "spawn") multiprocessing.set_start_method('fork') @njit(parallel=True) def func(x): return 10. * x arr = np.arange(2.) # run in single process to start Numba's thread pool np.testing.assert_allclose(func(arr), func.py_func(arr)) # now run in a multiprocessing pool to get a fork from a # non-main thread with multiprocessing.Pool(10) as p: result = p.map(func, [arr]) np.testing.assert_allclose(result, func.py_func(np.expand_dims(arr, 0))) assert threading_layer() == "workqueue" rPrrrrrQrrs r0test_workqueue_handles_fork_from_non_main_threadFTestMiscBackendIssues.test_workqueue_handles_fork_from_non_main_threadsO 6>>4/jjoo'2 #$#&   W &rr&N)r7r8r9r:rr(rrrrrr skipUnlessrrr;r&rrr~r~[s\F'',''0'+R(<=&'>&'rr~cZ\rSrSrSrSrSrSrSr\ S5r Sr S r \ S 5r S rg ) TestForkSafetyIssuesizN Checks Numba's behaviour in various situations involving GNU OpenMP and fork FcTSn[RSU/nURU5up4g)Nzsif 1: from numba.np.ufunc import omppool assert omppool.openmp_vendor == 'GNU' rP)rrr )r3rSrr+r s r!test_check_threading_layer_is_gnu6TestForkSafetyIssues.test_check_threading_layer_is_gnus->>4/<<(SrcSnURU-n[RSU/nURU5upEg![a%nUR S[ U55 SnAgSnAff=f)zf Whilst normally valid, this actually isn't for Numba invariant of OpenMP Checks SIGABRT is received. zif 1: X = np.arange(1000000.) Y = np.arange(1000000.) Z = busy_func(X, Y) pid = os.fork() if pid == 0: Z = busy_func(X, Y) else: os.wait() rPzfailed with code -6N)rKrrr rr-r)r3rRrSrr+r rzs r!test_par_parent_os_fork_par_child6TestForkSafetyIssues.test_par_parent_os_fork_par_childsf   $>>4/ 9||G,HC 9 MM/Q 8 8 9s: A)A$$A)cSnURU-n[RSU/nURU5upEUR(a [ XE5 gg)a- Implicit use of multiprocessing fork context. Does this: 1. Start with OpenMP 2. Fork to processes using OpenMP (this is invalid) 3. Joins fork 4. Check the exception pushed onto the queue that is a result of catching SIGTERM coming from the C++ aborting on illegal fork pattern for GNU OpenMP aif 1: mp = multiprocessing.get_context('fork') X = np.arange(1000000.) Y = np.arange(1000000.) q = mp.Queue() # Start OpenMP runtime on parent via parallel function Z = busy_func(X, Y, q) # fork() underneath with no exec, will abort proc = mp.Process(target = busy_func, args=(X, Y, q)) proc.start() err = q.get() assert "Caught SIGTERM" in str(err) rPNrKrrr r(r)r3rRrSrr+r s r*test_par_parent_implicit_mp_fork_par_child?TestForkSafetyIssues.test_par_parent_implicit_mp_fork_par_child sL   $>>4/<<( ;; #O rcSnURU-n[RSU/nURU5upEUR(a [ XE5 gg)a- Explicit use of multiprocessing fork context. Does this: 1. Start with OpenMP 2. Fork to processes using OpenMP (this is invalid) 3. Joins fork 4. Check the exception pushed onto the queue that is a result of catching SIGTERM coming from the C++ aborting on illegal fork pattern for GNU OpenMP aif 1: X = np.arange(1000000.) Y = np.arange(1000000.) ctx = multiprocessing.get_context('fork') q = ctx.Queue() # Start OpenMP runtime on parent via parallel function Z = busy_func(X, Y, q) # fork() underneath with no exec, will abort proc = ctx.Process(target = busy_func, args=(X, Y, q)) proc.start() proc.join() err = q.get() assert "Caught SIGTERM" in str(err) rPNrrs r*test_par_parent_explicit_mp_fork_par_child?TestForkSafetyIssues.test_par_parent_explicit_mp_fork_par_child-sL " $>>4/<<( ;; #O rcSnURU-n[RSU/nURU5upEUR(a [ XE5 gg)z Explicit use of multiprocessing spawn, this is safe. Does this: 1. Start with OpenMP 2. Spawn to processes using OpenMP 3. Join spawns 4. Run some more OpenMP aif 1: X = np.arange(1000000.) Y = np.arange(1000000.) ctx = multiprocessing.get_context('spawn') q = ctx.Queue() # Start OpenMP runtime and run on parent via parallel function Z = busy_func(X, Y, q) procs = [] for x in range(20): # start a lot to try and get overlap ## fork() + exec() to run some OpenMP on children proc = ctx.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) sys.stdout.flush() sys.stderr.flush() proc.start() [p.join() for p in procs] try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") # Run some more OpenMP on parent Z = busy_func(X, Y, q) rPNrrs r-test_par_parent_mp_spawn_par_child_par_parentBTestForkSafetyIssues.test_par_parent_mp_spawn_par_child_par_parentPsL : $>>4/<<( ;; #O rcSnURU-n[RSU/nURU5upEUR(a [ XE5 gg)z Implicit use of multiprocessing (will be fork, but cannot declare that in Py2.7 as there's no process launch context). Does this: 1. Start with no OpenMP 2. Fork to processes using OpenMP 3. Join forks 4. Run some OpenMP a,if 1: X = np.arange(1000000.) Y = np.arange(1000000.) q = multiprocessing.Queue() # this is ok procs = [] for x in range(10): # fork() underneath with but no OpenMP in parent, this is ok proc = multiprocessing.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) proc.start() [p.join() for p in procs] # and this is still ok as the OpenMP happened in forks Z = busy_func(X, Y, q) try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") rPNrrs r=test_serial_parent_implicit_mp_fork_par_child_then_par_parentRTestForkSafetyIssues.test_serial_parent_implicit_mp_fork_par_child_then_par_parent|L 2 $>>4/<<( ;; #O rcSnURU-n[RSU/nURU5upEUR(a [ XE5 gg)z Explicit use of multiprocessing 'fork'. Does this: 1. Start with no OpenMP 2. Fork to processes using OpenMP 3. Join forks 4. Run some OpenMP aif 1: X = np.arange(1000000.) Y = np.arange(1000000.) ctx = multiprocessing.get_context('fork') q = ctx.Queue() # this is ok procs = [] for x in range(10): # fork() underneath with but no OpenMP in parent, this is ok proc = ctx.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) proc.start() [p.join() for p in procs] # and this is still ok as the OpenMP happened in forks Z = busy_func(X, Y, q) try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") rPNrrs r=test_serial_parent_explicit_mp_fork_par_child_then_par_parentRTestForkSafetyIssues.test_serial_parent_explicit_mp_fork_par_child_then_par_parentrrr&N)r7r8r9r:rr(rrrr rrrrr;r&rrrrsSF)9,B  D*X'R&&rrc8\rSrSrSr\S5r\S5rSrg)TestTBBSpecificIssuesiFcSn[RSU/nURU5up4SnURXT5 UR(a[ SU5 [ SU5 gg)Na%if 1: import threading import numba numba.config.THREADING_LAYER='tbb' from numba import njit, prange, objmode from numba.core.serialize import PickleCallableByPath import os e_running = threading.Event() e_proceed = threading.Event() def indirect_core(): e_running.set() # wait for forker() to have forked while not e_proceed.isSet(): pass indirect = PickleCallableByPath(indirect_core) @njit def obj_mode_func(): with objmode(): indirect() @njit(parallel=True, nogil=True) def work(): acc = 0 for x in prange(10): acc += x obj_mode_func() return acc def runner(): work() def forker(): # wait for the jit function to say it's running while not e_running.isSet(): pass # then fork os.fork() # now fork is done signal the runner to proceed to exit e_proceed.set() numba_runner = threading.Thread(target=runner,) fork_runner = threading.Thread(target=forker,) threads = (numba_runner, fork_runner) for t in threads: t.start() for t in threads: t.join() rPz9Attempted to fork from a non-main thread, the TBB libraryOUT:ERR:)rrr r-r(r))r3rSrr+r msg_heads rtest_fork_from_non_main_thread4TestTBBSpecificIssues.test_fork_from_non_main_threads]64 l>>4/<<(N h$ ;; &#  &#  rcUR5 SnSRSU5n[RSU/n[R R 5nSUS'URX4S9upVX;aURS5 OURS U5 UR(a[S U5 [S U5 gg) NzSKIP: COMPILATION FAILEDax if 1: import ctypes import sys import multiprocessing as mp from tempfile import TemporaryDirectory, NamedTemporaryFile from numba.pycc.platform import Toolchain, external_compiler_works from numba import njit, prange, threading_layer import faulthandler faulthandler.enable() if not external_compiler_works(): raise AssertionError('External compilers are not found.') with TemporaryDirectory() as tmpdir: with NamedTemporaryFile(dir=tmpdir) as tmpfile: try: src = """ #define TBB_PREVIEW_WAITING_FOR_WORKERS 1 #include static tbb::task_scheduler_handle tsh; extern "C" { void launch(void) { tsh = tbb::task_scheduler_handle::get(); } } """ cxxfile = f"{tmpfile.name}.cxx" with open(cxxfile, 'wt') as f: f.write(src) tc = Toolchain() object_files = tc.compile_objects([cxxfile,], output_dir=tmpdir) dso_name = f"{tmpfile.name}.so" tc.link_shared(dso_name, object_files, libraries=['tbb',], export_symbols=['launch']) # Load into the process, it doesn't matter whether the # DSO exists on disk once it's loaded in. DLL = ctypes.CDLL(dso_name) except Exception as e: # Something is broken in compilation, could be one of # many things including, but not limited to: missing tbb # headers, incorrect permissions, compilers that don't # work for the above print(e) print('BROKEN_COMPILERS') sys.exit(0) # Do the test, launch this library and also execute a # function with the TBB threading layer. DLL.launch() @njit(parallel=True) def foo(n): acc = 0 for i in prange(n): acc += i return acc foo(1) # Check the threading layer used was TBB assert threading_layer() == 'tbb' # Use mp context for a controlled version of fork, this triggers the # reported bug. ctx = mp.get_context('fork') def nowork(): pass p = ctx.Process(target=nowork) p.start() p.join(10) print("SUCCESS") BROKEN_COMPILERSrPrrrQz3Compilation of DSO failed. Check output for detailsSUCCESSrr) skip_if_no_external_compilerreplacerrrrrr rr-r(r))r3rrSrrr+r s r&test_lifetime_of_task_scheduler_handle>4/jjoo', #$<<<1  " MMO P MM)S ) ;; &#  &#  rr&N) r7r8r9r:r(r rrr;r&rrrrs4FXXt__rrc4\rSrSrSrSr\S5rSrSr g)TestInitSafetyIssuesiFc[R"U[R[RS9n[R"[ UR 5nUR5 UR5upEURS:wa,[SUR<SUR5<S35eUR5 UR5UR54$!UR5 f=f)N)rrrrrrr)r3rrr r+r s rr TestInitSafetyIssues.run_cmds  (2(2://-<  MMO((*HC1$$%%szz|566% NN zz|SZZ\)) NN s AC((C:cH[RR[RR[5S5n[ R U/nURU5up4URSU5 UR(a[SU5 [SU5 gg)Nzorphaned_semaphore_usecase.pyzleaked semaphorerr) rrGrrHrIrrr  assertNotInr(r))r3 test_filerr+r s rtest_orphaned_semaphore,TestInitSafetyIssues.test_orphaned_semaphoresz GGLL!:!@B >>9-<<( +S1 ;; &#  &#  rcHSHn[R"U5 Sn[RSUR U5/nUR U5upEUR(a[SU5 [SU5 URX5 M g![a Mf=f)N)rrrzimport numba; import multiprocessing;multiprocessing.set_start_method('{}');print(multiprocessing.get_context().get_start_method())rPrr) rrrrrformatr r(r)r-)r3methrarr+r s rtest_lazy_lock_init(TestInitSafetyIssues.test_lazy_lock_inits4D ++D1MC~~tSZZ-=>G||G,HC{{fc"fc" MM$ $4   sB B! B!r&N) r7r8r9r:r(r r rrr;r&rrrrs'F*" %rrc\rSrSrSrSrg)TestOpenMPVendorsic[5nSUS'SUS'SUS'UR5HKn[RR U5(dM)UR X[ R5 MM g)z. Checks the OpenMP vendor strings are correct MSwin32Intelrrr=N)dictr>rr?r@ assertEqualr openmp_vendor)r3rGks r test_vendorsTestOpenMPVendors.test_vendorssh6 $!A||&&q))  g.C.CD!rr&N)r7r8r9r:rr;r&rrrrs Err__main__)crrmrfrrr`r*rrr^rrnumpyrnumbarrrrnumba.tests.supportrrr r r r rkt_queuenumba.testing.mainr _RUNNER_TIMEOUT numba.corerrnumba.np.ufunc.parallelrnumba.np.ufuncr_HAVE_TBB_POOL ImportErrorr_HAVE_OMP_POOLscipy.linalg.cython_lapackscipy _HAVE_LAPACKrrrr_gnuompskip_unless_gnu_ompr?r@_windows_osxmaxsize_32bitrrrr#r)r,objectr.r=rNrXr]rer{rThread _thread_classrrrrrrrrrTHREADING_LAYER_specific_backendsrrrr rCrMrZr~rrrrr7mainr&rrrs  >>GG: #% E!#&N&N%L !!.2NO !!.2KL  ;W22e;))'3JK << " "5 ) ||x(   $  v 22 & 2H22x222(  ,v,mW]]; !?7#;<$of&=>N*OL,IJ$_Y%?@UEhUEp++/JJ ')GH+1+I+B/v/>5A*,C5Ap3*H3*l"0"7"0"0J$$&C'!6C'C'LF'1F'F'Td0ddN 1 D9%89%9%x EE E$ z MMOy%N N Ls6%L ;LL+ LLL('L(+L76L7