Skip to content

Commit 38b509f

Browse files
christian-pintochristian-pinto
committed
Initial ThymesisFlow commit
This commit contains the whole Verilog codebase of ThymesisFlow including documentation on how to build, bringup and test our prototype design. Signed-off-by: Christian Pinto <[email protected]>
1 parent 0c20281 commit 38b509f

File tree

172 files changed

+57404
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

172 files changed

+57404
-0
lines changed

‎README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Welcome to the Home of ThymesisFlow
2+
Thymesisflow is a HW-SW co-designed prototype enabling
3+
hardware disaggregation of compute resources on POWER9 processor systems.
4+
The current design supports disaggregation of memory by "stealing" it from a
5+
neighbour node, and is based on OpenCAPI.
6+
7+
![ThymesisFlow Architecture](./reference_design_doc/thymesis_arch.jpg)
8+
9+
The ThymesisFlow approach consists of a compute node (left side of the figure)
10+
that is stealing the memory, and a memory node (righ side of the figure) from
11+
which the memory is stolen. The compute endpoint of the ThymesisFlow design is
12+
based on the OpenCAPI M1 mode (or LPC), while the memory endpointi uses the
13+
OpenCAPI C1 mode. On the compute endpoint the disaggregated memory is mapped at
14+
a specific range of addresses in the physical address space and can be
15+
dynamically hotplugged to a running Linux system. No software modification is
16+
needed to access disaggregated memory, neither in the Linux kernel nor from the
17+
user applications.
18+
19+
This repository contains the complete design (Verilog/HLS) of memory and compute
20+
side, including the OpenCAPI 3.0 reference design.
21+
22+
Please refer to the [ThymesisFlow documentation](./reference_design_doc/ThymesisFlow%20Design%20and%20Deployment%20v1.1.pdf) for a complete description of the
23+
design and its bringup.
24+
25+
## Supported Cards
26+
27+
The ThymesisFlow design currently supports the following cards:
28+
29+
- [Alphadata 9V3](https://www.alpha-data.com/dcp/products.php?product=adm-pcie-9v3)
30+
31+
## How-to bring-up the design
32+
The ThymesisFlow build and bring-up procedure is described in detail in the [design reference manual](./reference_design_doc/ThymesisFlow%20Design%20and%20Deployment%20v1.1.pdf).
33+
Please, refer also to the [OpenCAPI3.0
34+
Wiki](https://github.com/OpenCAPI/OpenCAiPI3.0_Client_RefDesign/wiki) for further details on the Vivado projects creation.
35+
36+
### Create Compute side Vivado project
37+
```console
38+
vivado -source create_project.tcl -tclargs --speed 20.0 --afu thymesisflow --buffer bypass --tftype compute
39+
```
40+
41+
### Create Memory side Vivado project
42+
```console
43+
vivado -source create_project.tcl -tclargs --speed 20.0 --afu thymesisflow --buffer bypass --tftype memory
44+
```
45+
46+
## Resources:
47+
48+
- [ThymesisFlow at the OpenPOWER Summit North America 2019](https://www.youtube.com/watch?v=XcjRL3Lh8Ig)
49+
- [ThymesisFlow at the OpenPOWER Summit Europe 2018](https://www.youtube.com/watch?v=vSKUeGeEkoA)
50+
51+
## What's Next
52+
53+
Our immediate next steps are related to the release of the software support for
54+
ThymesisFlow:
55+
56+
- Memory stealing process: this is the user application that allocates a buffer in
57+
the memory node for access from a remote node.
58+
- ThymesisStats: Software Library for accessing performance countersi embedded in the design.
59+
- ThymesisFlow orchestration software: we gave a stub at our vision of a
60+
disaggregated systemi, and came up with a proposal control plane for
61+
dynamic orchestration of disaggregated memory segments.
62+
- Stay tuned!

‎afu/thymesisflow/aurora_global_poweron.v

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
// *!***************************************************************************
2+
// *! Copyright 2019 International Business Machines
3+
// *!
4+
// *! Licensed under the Apache License, Version 2.0 (the "License");
5+
// *! you may not use this file except in compliance with the License.
6+
// *! You may obtain a copy of the License at
7+
// *! http://www.apache.org/licenses/LICENSE-2.0
8+
// *!
9+
// *! The patent license granted to you in Section 3 of the License, as applied
10+
// *! to the "Work," hereby includes implementations of the Work in physical form.
11+
// *!
12+
// *! Unless required by applicable law or agreed to in writing, the reference design
13+
// *! distributed under the License is distributed on an "AS IS" BASIS,
14+
// *! WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
// *! See the License for the specific language governing permissions and
16+
// *! limitations under the License.
17+
// *!
18+
// *! The background Specification upon which this is based is managed by and available from
19+
// *! the OpenCAPI Consortium. More information can be found at https://opencapi.org.
20+
// *!***************************************************************************
21+
// Module designer: Dimitris Syrivelis
22+
// Backup: Christian Pinto, Michele Gazzetti
23+
24+
`timescale 1ns / 10ps
25+
26+
// ==============================================================================================================================
27+
// @@@ Module Declaration
28+
// ==============================================================================================================================
29+
module aurora_power_on_reset (
30+
input clock // Clock - needs to be the aurora init_clk and needs to be stable.
31+
, input reset_n // needs to be active high early reset (ocde - would be fine)
32+
, input sys_out_aurora_r // Let's you know when to deassert rb
33+
, output pma_init // locally registered to meet timing.
34+
, output rpb //
35+
, output rout_n
36+
) ;
37+
38+
reg pos_pma_init_q;
39+
reg pos_reset_pb_q;
40+
reg pos_reset_logic_n_q;
41+
42+
reg pow_pma_init_q;
43+
reg pow_reset_pb_q;
44+
reg pow_reset_logic_n_q;
45+
46+
reg ce_pma_init_q;
47+
reg ce_reset_pb_q;
48+
reg ce_reset_logic_n_q;
49+
50+
reg dsor_pma_init_q;
51+
reg dsor_reset_pb_q;
52+
reg dsor_reset_logic_n_q;
53+
54+
reg oor_pma_init_q;
55+
reg oor_reset_pb_q;
56+
reg oor_reset_logic_n_q;
57+
58+
59+
reg[7:0] cycle_counter_poweron;
60+
reg[7:0] cycle_counter_poweron2;
61+
//These regs should go to the core
62+
assign pma_init = pos_pma_init_q | pow_pma_init_q | ce_pma_init_q | dsor_pma_init_q | oor_pma_init_q;
63+
assign rpb = pos_reset_pb_q | pow_reset_pb_q | ce_reset_pb_q | dsor_reset_pb_q | oor_reset_pb_q;
64+
assign rout_n = pos_reset_logic_n_q | pow_reset_logic_n_q | ce_reset_logic_n_q | dsor_reset_logic_n_q | oor_reset_logic_n_q;
65+
66+
reg [2:0] SM;
67+
parameter POWER_ON_SEQ = 3'b000;
68+
parameter POWER_ON_WAIT = 3'b001;
69+
parameter COUNTER_EXPIRED = 3'b010;
70+
parameter DETECT_SYS_OUT_RESET_HIGH = 3'b011;
71+
parameter OUT_OF_RESET = 3'b100;
72+
73+
always @(posedge(clock))
74+
if (SM == POWER_ON_SEQ)
75+
begin
76+
pos_reset_logic_n_q <= 1'b0;//Activate reset
77+
pos_pma_init_q <= 1'b1; //assert active low pma_init_q for power_on
78+
pos_reset_pb_q <= 1'b1; //assert pma_init_q for
79+
cycle_counter_poweron <= 8'h20;
80+
end
81+
else
82+
begin
83+
cycle_counter_poweron <= cycle_counter_poweron - 8'h01;
84+
pos_reset_logic_n_q <= 1'b0;//invalidate for or gate
85+
pos_pma_init_q <= 1'b0; //invalidate for or gate
86+
pos_reset_pb_q <= 1'b0; //invalidatae for or gate
87+
end
88+
89+
90+
always @(posedge(clock))
91+
if (SM == POWER_ON_WAIT)
92+
begin
93+
pow_reset_logic_n_q <= 1'b0;//maintain reset
94+
pow_pma_init_q <= 1'b1; //maintain assert pma_init_q for power_on
95+
pow_reset_pb_q <= 1'b1; //maintain assert pma_init_q for
96+
end
97+
else
98+
begin
99+
pow_reset_logic_n_q <= 1'b0;
100+
pow_pma_init_q <= 1'b0;
101+
pow_reset_pb_q <= 1'b0;
102+
end
103+
104+
always @(posedge(clock))
105+
if (SM == COUNTER_EXPIRED)
106+
begin
107+
ce_reset_logic_n_q <= 1'b0;//maintain reset
108+
ce_pma_init_q <= 1'b0; //deassert pma_init_q for power_on
109+
ce_reset_pb_q <= 1'b1; //maintain assert pma_init_q for
110+
cycle_counter_poweron2 <= 8'h20;
111+
end
112+
else
113+
begin
114+
cycle_counter_poweron2 <= cycle_counter_poweron2 - 8'h01;
115+
ce_reset_logic_n_q <= 1'b0;
116+
ce_pma_init_q <= 1'b0;
117+
ce_reset_pb_q <= 1'b0;
118+
end
119+
120+
always @(posedge(clock))
121+
if (SM == DETECT_SYS_OUT_RESET_HIGH)
122+
begin
123+
dsor_reset_logic_n_q <= 1'b0;//maintain reset
124+
dsor_pma_init_q <= 1'b0; //maintain deassert pma_init_q for power_on
125+
dsor_reset_pb_q <= 1'b1; //maintain assert
126+
end
127+
else
128+
begin
129+
dsor_reset_logic_n_q <= 1'b0;
130+
dsor_pma_init_q <= 1'b0;
131+
dsor_reset_pb_q <= 1'b0;
132+
end
133+
134+
always @(posedge(clock))
135+
if(SM == OUT_OF_RESET)
136+
begin
137+
oor_reset_logic_n_q <= 1'b1;//deassert reset (active low)
138+
oor_pma_init_q <= 1'b0; //maintain deassert pma_init_q for power_on
139+
oor_reset_pb_q <= 1'b0; //deassert rb
140+
end
141+
else
142+
begin
143+
oor_reset_logic_n_q <= 1'b0;
144+
oor_pma_init_q <= 1'b0;
145+
oor_reset_pb_q <= 1'b0;
146+
end
147+
148+
//State machine handling
149+
always @(posedge(clock))
150+
if (reset_n == 1'b0)
151+
begin
152+
SM <= POWER_ON_SEQ;
153+
end
154+
else
155+
begin
156+
case (SM)
157+
POWER_ON_SEQ:
158+
SM <= POWER_ON_WAIT;
159+
POWER_ON_WAIT:
160+
if (cycle_counter_poweron == 8'h0)
161+
SM <= COUNTER_EXPIRED;
162+
else SM <= POWER_ON_WAIT;
163+
COUNTER_EXPIRED:
164+
SM <= DETECT_SYS_OUT_RESET_HIGH;
165+
DETECT_SYS_OUT_RESET_HIGH:
166+
if (cycle_counter_poweron2 == 8'h0)
167+
SM <= OUT_OF_RESET;
168+
else SM <= DETECT_SYS_OUT_RESET_HIGH;
169+
OUT_OF_RESET:
170+
SM <= OUT_OF_RESET; //lock in this state permanently.
171+
endcase
172+
end
173+
endmodule

‎afu/thymesisflow/hls_modules/memory_egress/build_hls_module.tcl

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Create a project
2+
open_project -reset ocx_memory_egress
3+
4+
# The source files
5+
add_files ocx_memory_egress_top.cpp
6+
add_files ocx_flit.cpp
7+
add_files ocx_resp_eng.cpp
8+
add_files ocx_rty_eng.cpp
9+
10+
# Specify the top-level function for synthesis
11+
set_top ocx_memory_egress_top
12+
13+
###########################
14+
# Solution settings
15+
16+
# Create solution1
17+
open_solution -reset solution1
18+
19+
set_part {xcvu3p-ffvc1517-2-i}
20+
create_clock -period 2.5
21+
config_rtl -reset_level low
22+
23+
## Synthesize design
24+
csynth_design
25+
## Export for vivado IP catalog integration
26+
export_design -rtl verilog -format ip_catalog -description "ThymesisFlow Memory Egress module" -vendor "ibm.com" -library "ThymesisFlow" -display_name "ocx_memory_egress"
27+
28+
exit
29+

‎afu/thymesisflow/hls_modules/memory_egress/globals.hpp

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
// *!***************************************************************************
2+
// *! Copyright 2019 International Business Machines
3+
// *!
4+
// *! Licensed under the Apache License, Version 2.0 (the "License");
5+
// *! you may not use this file except in compliance with the License.
6+
// *! You may obtain a copy of the License at
7+
// *! http://www.apache.org/licenses/LICENSE-2.0
8+
// *!
9+
// *! The patent license granted to you in Section 3 of the License, as applied
10+
// *! to the "Work," hereby includes implementations of the Work in physical form.
11+
// *!
12+
// *! Unless required by applicable law or agreed to in writing, the reference design
13+
// *! distributed under the License is distributed on an "AS IS" BASIS,
14+
// *! WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
// *! See the License for the specific language governing permissions and
16+
// *! limitations under the License.
17+
// *!
18+
// *! The background Specification upon which this is based is managed by and available from
19+
// *! the OpenCAPI Consortium. More information can be found at https://opencapi.org.
20+
// *!***************************************************************************
21+
// Module designer: Dimitris Syrivelis
22+
// Backup: Christian Pinto, Michele Gazzetti
23+
24+
#ifndef GLOBALS_HH
25+
#define GLOBALS_HH
26+
27+
#include <stdio.h>
28+
#include <iostream>
29+
#include <fstream>
30+
#include <string>
31+
#include <math.h>
32+
#include <stdint.h>
33+
#include <cstdlib>
34+
#include <hls_stream.h>
35+
#include <ap_int.h>
36+
37+
using namespace hls;
38+
39+
#define TL_RD_RESPOK 0x4
40+
#define TL_RD_RESPFAIL 0x5
41+
#define TL_WR_RESPOK 0x8
42+
#define TL_WR_RESPFAIL 0x9
43+
#define TL_XLATE_DONE 0x18
44+
45+
46+
#define TLRESP_DATA 0x2
47+
#define TLRESP_CODE_RTY 0x2
48+
#define TLRESP_CODE_PEND 0x4
49+
50+
#define TLRESP_RTY 0x1
51+
#define TLRESP_DATA 0x2
52+
53+
#define ADDR_MASK 0xc000
54+
#define LOOKUPSIZE 512
55+
56+
#endif

0 commit comments

Comments
 (0)