Optimize the instruction fetch path

[yy214123]

This improves the instruction-fetch path by adding new cache structures and enhancing the observability of fetch behavior.

Key Improvements

1. Replaces the previous 1-entry direct-mapped design with a 2-entry hash-indexed TLB.
2. Adds a direct-mapped instruction cache that stores recently fetched instructions and provides a fast hit path before triggering address translation.
3. Provides finer-grained fetch statistics to support and validate the above architectural changes.

Here’s the updated performance comparison:

	1-entry TLB	2-entry TLB	+ 64KB Direct-Mapped I-cache
Fetch hit	95,971,425	96,757,879	97,706,775
Fetch miss	4,028,576	3,242,122 (-19.5%)	1,026,354 (-68%)
Fetch hit rate	95.97%	96.76%	98.96%
Load hit rate	93.15%	93.15%	93.15%
Store hit rate	95.57%	95.57%	95.57%

[jserv]

Unify the naming scheme.

[jserv]

Rebase the latest 'master' branch and resolve build errors.

[visitorckw]

This series appears to contain several "fix-up," "refactor," or "build-fix" commits that correct or adjust a preceding patch.

To maintain a clean history and ensure the project is bisectable, each patch in a series should be complete and correct on its own.

[visitorckw]

As a friendly reminder regarding project communication:

Please ensure that when you quote-reply to others' comments, you do not translate the quoted text into any language other than English.

This is an open-source project, and it's important that we keep all discussions in English. This ensures that the conversation remains accessible to everyone in the community, including current and future participants who may not be familiar with other languages.

[visitorckw]

This code looks suspicious to me.

When you move the evicted I-cache block (tmp) back into the victim cache, you are setting the vblk->tag to tmp.tag, which is the 16-bit I-cache tag.

Won't this corrupts the victim cache entry? The VC search logic requires a 24-bit tag ([ICache Tag | ICache Index]) to function. Because you're only storing the 16-bit tag, this VCache entry will never be hit again.

[yy214123]

Won't this corrupts the victim cache entry? The VC search logic requires a 24-bit tag ([ICache Tag | ICache Index]) to function. Because you're only storing the 16-bit tag, this VCache entry will never be hit again.

Thank you for pointing that out. I’ve added the following expressions to ensure correctness :

+   vblk->tag = (tmp.tag << ICACHE_INDEX_BITS) | idx;

[jserv]

Found that the victim cache implementation still has significant room for improvement:

What are you going to proceed with?

[yy214123]

What are you going to proceed with?

Not sure yet if it’s due to the FIFO policy or the victim cache size.
I’ll try a simple approximate LRU version (with a counter) to see how much the replacement policy affects the stats.

Does this approach make sense to you, or would you suggest a different direction?

[jserv]

Does this approach make sense to you, or would you suggest a different direction?

When in doubt, measure before you choose a method.

[yy214123]

When in doubt, measure before you choose a method.

I ran some benchmarks on this new LRU setup, with all cache statistics generated after executing 100 million instructions to ensure a fair comparison. Here's what I found.

With the same 256 I-Cache / 16 V-Cache blocks, just switching FIFO to LRU only gave a slight hit rate boost:

	LRU	FIFO
victim cache hit rate	4.16%	4.10%
victim cache miss rate	95.84%	95.90%

I then doubled the V-Cache (16 -> 32 blocks), but the miss rate was still really high:

	LRU	FIFO
victim cache hit rate	5.74%	5.68%
victim cache miss rate	94.26%	94.32%

This means the misses hitting the V-Cache are almost all compulsory misses, not the conflict misses , to prove this, I intentionally shrank the I-Cache (256 -> 64) to force more conflicts.

	LRU	FIFO
victim cache hit rate	11.85%	11.64%
victim cache miss rate	88.15%	88.36%

The LRU logic is working fine. The key takeaway is that compared to FIFO, we did see a tiny improvement, which confirms the LRU strategy successfully handled the few conflict misses that were present.

[jserv]

The measurements reveal that 98% of the benefit comes from a simple 2-entry TLB cache, not from the elaborate 64KB I-cache + 16-block victim cache with LRU. The working set fits comfortably in 64KB. The victim cache is solving a problem that doesn't exist. These are compulsory misses, not conflict misses.

LRU Overhead Cost:

  // O(N) linear scan on EVERY I-cache miss (6.19M times):
  for (int i = 1; i < 16; i++) {  // 16 comparisons each
      if (vm->icache.v_used[i] < lru_min) { ... }
  }

• 6.19M misses × 16 comparisons = 99M operations
• Saves only 405K victim cache hits

[jserv]

Pointer aliasing time bomb:

• I-cache stores blk->base pointing to physical memory
• TLB stores page_addr pointing to same memory
• These can become desynchronized on page remapping → potential use-after-free

[yy214123]

Does this mean that I also need to add I-cache and victim-cache invalidation inside the mmu_invalidate_range?

[yy214123]

I have added the relevant code in mmu_invalidate_range:

for (int i = 0; i < ICACHE_BLOCKS; i++) {
    icache_block_t *blk = &vm->icache.block[i];
    if (!blk->valid)
        continue;

    uint32_t icache_vpn = (blk->tag << ICACHE_INDEX_BITS) | i;
    icache_vpn >>= (RV_PAGE_SHIFT - ICACHE_OFFSET_BITS);
    if (icache_vpn >= start_vpn && icache_vpn <= end_vpn)
        blk->valid = false;
}

[yy214123]

After discussing with @jserv , we decided to remove the victim cache since the benefit was limited and the implementation added unnecessary complexity.
I also reordered the commits to ensure the 2-entry TLB results aren’t affected by the I-cache changes.

Here’s the updated performance comparison:

	1-entry TLB	2-entry TLB	+ 64KB Direct-Mapped I-cache
Fetch hit	95,971,425	96,757,879	97,706,775
Fetch miss	4,028,576	3,242,122 (-19.5%)	1,026,354 (-68%)
Fetch hit rate	95.97%	96.76%	98.96%
Load hit rate	93.15%	93.15%	93.15%
Store hit rate	95.57%	95.57%	95.57%

[jserv]

we decided to remove the victim cache since the benefit was limited and the implementation added unnecessary complexity. I also reordered the commits to ensure the 2-entry TLB results aren’t affected by the I-cache changes.

Update the descriptions of this pull request.

[yy214123]

Update the descriptions of this pull request.

I have updated the title and description at the top.

[jserv]

Is this pull request ready to proceed?

[yy214123]

Is this pull request ready to proceed?

Yes, I am currently keeping only the simple implementations that can effectively bring improvements.

[cubic-dev-ai]

1 issue found across 3 files

Prompt for AI agents (all 1 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="main.c">

<violation number="1" location="main.c:1067">
P3: Per-hart cache statistics no longer include the hart index, so multi-hart output is ambiguous and you can’t tell which hart each block refers to.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[cubic-dev-ai]

P3: Per-hart cache statistics no longer include the hart index, so multi-hart output is ambiguous and you can’t tell which hart each block refers to.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At main.c, line 1067:

<comment>Per-hart cache statistics no longer include the hart index, so multi-hart output is ambiguous and you can’t tell which hart each block refers to.</comment>

<file context>
@@ -1047,14 +1063,25 @@ static void print_mmu_cache_stats(vm_t *vm)
-                    100.0 * hart-&gt;cache_fetch.hits / fetch_total);
-        fprintf(stderr, &quot;\n&quot;);
 
+        fprintf(stderr, &quot;\n=== Introduction Cache Statistics ===\n&quot;);
+        fprintf(stderr, &quot;  Total access:  %12llu\n&quot;, access_total);
+        if (access_total &gt; 0) {
</file context>

Suggested change

	fprintf(stderr, "\n=== Introduction Cache Statistics ===\n");
	fprintf(stderr, "\nHart %u:\n=== Instruction Cache Statistics ===\n", i);

[yy214123]

I have added the relevant code.
After compiling using $make SMP=2 check, the following statistics are output:

=== MMU Cache Statistics ===

Hart 0:

=== Introduction Cache Statistics ===
  Total access:     358547968
  Icache hits:      349789985 (97.56%)
  Icache misses:      8757983 (2.44%)
   ├ TLB hits:        4108502 (46.91%)
   └ TLB misses:      4649481 (53.09%)

=== Data Cache Statistics ===
  Load:      74468422 hits,      4645526 misses (8x2) (94.13% hit rate)
  Store:     60833790 hits,      1337401 misses (8x2) (97.85% hit rate)

Hart 1:

=== Introduction Cache Statistics ===
  Total access:     313707200
  Icache hits:      305442748 (97.37%)
  Icache misses:      8264452 (2.63%)
   ├ TLB hits:        3563779 (43.12%)
   └ TLB misses:      4700673 (56.88%)

=== Data Cache Statistics ===
  Load:      68932725 hits,      3193502 misses (8x2) (95.57% hit rate)
  Store:     50140013 hits,       754139 misses (8x2) (98.52% hit rate)

[jserv]

Thank @yy214123 for contributing!