gevent 1.3.2 on Windows fails

[nedbat]

See: gevent/gevent#1232

================================== FAILURES ===================================
_________________________ ConcurrencyTest.test_gevent _________________________
self = <tests.test_concurrency.ConcurrencyTest testMethod=test_gevent>
    def test_gevent(self):
        code = (GEVENT + SUM_RANGE_Q + PRINT_SUM_RANGE).format(QLIMIT=self.QLIMIT)
>       self.try_some_code(code, "gevent", gevent)
tests\test_concurrency.py:268: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests\test_concurrency.py:248: in try_some_code
    self.assertEqual(data.line_counts()['try_it.py'], lines)
E   AssertionError: 43 != 37
---------------------------- Captured stdout call -----------------------------
499500
    from gevent import monkey
    monkey.patch_thread()
    import threading
    import gevent.queue as queue
    
    # Above this will be imports defining queue and threading.
    class Producer(threading.Thread):
        def __init__(self, limit, q):
            threading.Thread.__init__(self)
            self.limit = limit
            self.q = q
        def run(self):
            for i in range(self.limit):
                self.q.put(i)
            self.q.put(None)
    class Consumer(threading.Thread):
        def __init__(self, q, qresult):
            threading.Thread.__init__(self)
            self.q = q
            self.qresult = qresult
        def run(self):
            sum = 0
            while True:
                i = self.q.get()
                if i is None:
                    break
                sum += i
            self.qresult.put(sum)
    def sum_range(limit):
        q = queue.Queue()
        qresult = queue.Queue()
        c = Consumer(q, qresult)
        p = Producer(limit, q)
        c.start()
        p.start()
        p.join()
        c.join()
        return qresult.get()
    # Below this will be something using sum_range.
    
    print(sum_range(1000))
    
43: [2, 3, 4, 5, 9, 10, 15, 20, 21, 26, 35, 49, 36, 37, 38, 22, 23, 24, 39, 11, 12, 13, 40, 27, 28, 29, 472, 477, 431, 432, 433, 434, 41, 16, 17, 18, 43, 44, 30, 32, 31, 33, 45]
   
 X     from gevent import monkey
 X     monkey.patch_thread()
 X     import threading
 X     import gevent.queue as queue
       
       # Above this will be imports defining queue and threading.
   
 X     class Producer(threading.Thread):
 X         def __init__(self, limit, q):
 X             threading.Thread.__init__(self)
 X             self.limit = limit
 X             self.q = q
   
 X         def run(self):
 X             for i in range(self.limit):
 X                 self.q.put(i)
 X             self.q.put(None)
   
 X     class Consumer(threading.Thread):
 X         def __init__(self, q, qresult):
 X             threading.Thread.__init__(self)
 X             self.q = q
 X             self.qresult = qresult
   
 X         def run(self):
 X             sum = 0
 X             while True:
 X                 i = self.q.get()
 X                 if i is None:
 X                     break
 X                 sum += i
 X             self.qresult.put(sum)
   
 X     def sum_range(limit):
 X         q = queue.Queue()
 X         qresult = queue.Queue()
 X         c = Consumer(q, qresult)
 X         p = Producer(limit, q)
 X         c.start()
 X         p.start()
   
 X         p.join()
 X         c.join()
 X         return qresult.get()
   
       # Below this will be something using sum_range.
       
 X     print(sum_range(1000))
       
1 failed, 787 passed, 5 skipped in 133.48 seconds

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/663

[nedbat]

Original comment by Jason Madden (Bitbucket: jamadden, GitHub: jamadden)

---

I did some looking into this, covering the things I talked about on the github issue.

I'll be talking about the entry for try_it.py in the .coverage file created by coverage run --concurrency=gevent try_it.py.

First, on Windows with CPython 3.6, I get 43 entries using gevent 1.3.3:

[1,2,3,4,7,8,12,16,17,21,29,41,30,31,32,18,19,20,33,9,10,11,34,22,23,24,471,476,430,431,432,433,35,13,14,15,36,37,25,27,26,28,38]

It doesn't exactly match the output above, probably because of a leading blank line, but it does have length 43. What stands out are the sequence of entries in the 400 range: 471, 476, 430, 431, etc. There are 6 of them, the difference from the expected number.

When I run gevent in PURE_PYTHON mode, e.g., where Greenlet and Queue are not compiled with Cython, I get this output:

[1,2,3,4,7,8,12,16,17,21,29,41,30,31,32,18,19,20,33,9,10,11,34,22,23,24,35,13,14,15,36,37,25,27,26,28,38]

There are no numbers in the 400 range, and it has the expected length of 37.

I then edited gevent/queue.py to disable just the C extension for it (leaving Greenlet compiled) and ran coverage again on my Windows box machine. I got the expected length of 37 again.

So something about having queue.Queue() compiled with Cython is causing issues. The 400 series lines mentioned, BTW, are not particularly interesting ones in the queue source code, mostly being docstrings, and all being past the definition of the Queue class.

(There was a period of time where I thought I had reproduced the incorrect length on macOS, but I can't do it reliably, so...I dunno.)