Upgrade pyquarkchain from Python 3.8 to Python 3.13#969
Draft
Upgrade pyquarkchain from Python 3.8 to Python 3.13#969
Conversation
- adjust_difficulty.py: replace bare `import monitoring` with `from quarkchain.tools import monitoring` (breaks when run from outside the tools directory) - adjust_difficulty.py: replace jsonrpc_async.Server with AsyncJsonRpcClient and use .call() method - monitoring.py: replace jsonrpc_async.Server with AsyncJsonRpcClient and use .call() method; close session via .close() - jsonrpc_client.py: fix AsyncJsonRpcClient.call signature to use *params (variadic) to match JsonRpcClient.call, fixing callers that pass positional arguments Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix send_request in test_jsonrpc.py to unpack list params correctly instead of double-wrapping them (e.g. [["0x..."]] → ["0x..."]) - Add call_with_dict_params to AsyncJsonRpcClient for named params - Implement JSONRPCWebsocketServer.shutdown() to actually close the server, fixing test isolation hangs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Track and cancel all fire-and-forget asyncio tasks that were leaking across tests, causing resource exhaustion and timeout failures. - AbstractConnection: add _loop_task and _handler_tasks tracking; use try/finally in active_and_loop_forever to ensure cleanup on cancellation; cancel _loop_task in close() - master.py: track SlaveConnection loop task and __init_cluster task; cancel _init_task on shutdown - slave.py: track MasterConnection, SlaveConnection, PeerShardConnection loop tasks and __start_server task - shard.py: track PeerShardConnection loop task - miner.py: track and cancel mining task in disable() - simple_network.py: track Peer loop task and connect_seed task - test_utils.py: restructure shutdown_clusters with try/finally to guarantee task cleanup; await server.wait_closed() for slave servers - conftest.py: multi-round task cancellation; reset aborted_rpc_count
- Fix AttributeError: replace undefined self.external_port/self.internal_port/self.protocol with self.port - Fix _discover timeout: wrap async_search with asyncio.wait_for instead of waiting after it completes - Fix _run: guard _add_port_mapping with self._service check to avoid AttributeError - Fix _delete_port_mapping: delete both TCP and UDP mappings to match AddPortMapping - Remove dead code: _refresh_task and _running fields that were never used
- convert test_cluster.py and test_jsonrpc.py to unittest.IsolatedAsyncioTestCase - make create_test_clusters/shutdown_clusters async, ClusterContext async context manager - replace call_async() with await, assert_true_with_timeout with async_assert_true_with_timeout - replace _get_or_create_event_loop() with asyncio.get_running_loop() in master/slave - fix mock_pay_native_token_as_gas to support both sync and async wrapped functions - remove obsolete _get_or_create_event_loop, call_async, assert_true_with_timeout helpers - fix conftest to restore event loop after IsolatedAsyncioTestCase closes it
replace with Any type hint to avoid deprecation warning in websockets 14+
This reverts commit b4e190d.
Migrate test classes to IsolatedAsyncioTestCase so that asyncio.create_task calls in production code (shard_state, root_state) have a running event loop. - Replace asyncio.ensure_future with asyncio.create_task in production code - Replace _get_or_create_event_loop with asyncio.get_running_loop in master/slave - Convert ClusterContext to async context manager - Convert create_test_clusters / shutdown_clusters to async - Add fire_and_forget and async_assert_true_with_timeout utilities - Simplify conftest fixture to only restore event loop after IsolatedAsyncioTestCase teardown
…sues - Rename jsonrpcserver.py to jsonrpc_server.py for consistent naming - Replace armor with asyncio.shield for cancellation protection - Fix get_running_loop to get_event_loop for compatibility - Fix subscription.py import from old jsonrpcserver package - Fix indentation in test_jsonrpc.py and method name in stats.py
- Fix snake_case field names to camelCase for JSON-RPC responses (network_id->networkId, shard_size->chainSize, block_height->blockHeight, contract_address->contractAddress) - Fix resp.data.result access pattern to direct dict access - Add exception handling and cli.close() to prevent connection leaks - Add 0x prefix for address in balance_watcher query_balance
- extract shared socket/aiohttp mocks into fixtures - extract fake_wait helper to reduce duplication - add tests for edge cases: exception handling, missing service, session cleanup - fix coroutine never awaited warning in test_run_exits_on_cancel
…internal RPC errors
…arrays ethash_utils.py: - replace hex-based encode_int/decode_int with struct.pack/unpack for serialize_hash and deserialize_hash (~30x faster per call) - inline ethash_sha3_512/256 to skip intermediate list conversion (~5x faster on list input) - add ethash_sha3_512_np and ethash_sha3_256_np: numpy ndarray variants that accept bytes or ndarray and return uint32 ndarray, eliminating tolist()/ np.array() round-trips in the hot path - consolidate keccak implementation here; ethash.py no longer duplicates it ethash.py: - store cache as 2D numpy uint32 ndarray (shape n x 16) via _get_cache - use ethash_sha3_512_np/256_np throughout to keep data in ndarray form - vectorize the 16-element mix update in calc_dataset_item and hashimoto inner loop using numpy arithmetic instead of list(map(fnv, ...)) - scalar fnv for cache_index uses plain Python int to avoid numpy scalar overhead test_ethash.py: - add TestEthashUtils covering serialize_hash, deserialize_hash, fnv, ethash_sha3_512/256 directly against reference implementations Benchmark: hashimoto_light ~23% faster end-to-end vs pure Python baseline; serialize_hash/deserialize_hash ~30x faster individually
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dependencies (requirements.txt)
asyncio API migration
JSON-RPC library replacement
Python 3.12+ removed APIs
p2p module
Test stability
Code formatting
Test plan