Skip to content

Commit a3ed3a3

Browse files
brianjgeigerbrianjgeiger
authored
Merge pull request #1073 from futa-ikeda/feature/pbs-26-19
Merge develop into feature/pbs-26-19
2 parents bed4d25 + 63b6e33 commit a3ed3a3

23 files changed

Lines changed: 885 additions & 185 deletions

‎CHANGELOG‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
We follow the CalVer (https://calver.org/) versioning scheme: YY.MINOR.MICRO.
44

5+
26.20.0 (2026-10-01)
6+
====================
7+
* SSR Metrics
8+
59
26.19.0 (2026-09-23)
610
====================
711
* Remove leave this view from top of registration VOLs

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,14 @@ take up to 60 seconds once the docker build finishes.
2323
### Recommended
2424

2525
- [Docker Commands](docs/docker.md).
26+
- [Architecture](docs/arch.md).
2627
- [ESLint Strategy](docs/eslint.md).
2728
- [Git Conventions](docs/git-convention.md).
2829
- [i18n](docs/i18n.md).
30+
- [Models conventions](docs/models.md).
2931
- [NGXS Conventions](docs/ngxs.md).
32+
- [SSR](docs/ssr.md).
33+
- [SSR metrics](docs/ssr-metrics.md).
3034
- [Testing Strategy](docs/testing.md).
3135
- [Sentry error filtering](docs/sentry.md).
3236

@@ -60,6 +64,6 @@ Install Volta from [volta](https://volta.sh/) and it will automatically pin Node
6064

6165
## Configuration
6266

63-
OSF uses an `assets/config/config.json` file for any 3rd-party tokens. This file is not committed to the repo.
67+
OSF uses `src/assets/config/config.json` for third-party tokens and environment URLs. This file is not committed to the repo.
6468

65-
There is a `assets/config/template.json` file that can be copied to `assets/config/config.json` to store any 3rd-party tokens locally.
69+
Copy `src/assets/config/template.json` to `src/assets/config/config.json` for local development. At runtime the app loads it from `/assets/config/config.json`.

‎docs/arch.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,12 @@ See [NGXS State Management](./ngxs.md).
7979

8080
---
8181

82+
## SSR
83+
84+
Server-side rendering, route render modes, config, and bot traffic are documented in [SSR](./ssr.md). Render metrics are in [SSR metrics](./ssr-metrics.md).
85+
86+
---
87+
8288
## 🚀 Dynamic File Generation (Schematics)
8389

8490
Use Angular CLI for scaffolding:

‎docs/ssr-metrics.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# SSR metrics
2+
3+
## Index
4+
5+
- [Overview](#overview)
6+
- [What we track](#what-we-track)
7+
- [When a metric is sent](#when-a-metric-is-sent)
8+
- [When HTML is inspected](#when-html-is-inspected)
9+
- [is_complete rules](#is_complete-rules)
10+
- [content_type](#content_type)
11+
- [Payload and API](#payload-and-api)
12+
- [Related docs](#related-docs)
13+
14+
---
15+
16+
## Overview
17+
18+
After the SSR server sends HTML to a crawler, it POSTs a JSON:API metric to the OSF API. Collection runs in the background and does not delay the response.
19+
20+
Implementation: `src/server/ssr-metrics.middleware.ts`, `src/server/ssr-metrics.ts`, `src/server/ssr-html-metrics.ts`.
21+
22+
In production, the SSR Node process receives **bot traffic only**. Metrics are emitted for HTML navigations on that server.
23+
24+
---
25+
26+
## What we track
27+
28+
| Goal | Field | Notes |
29+
| ------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
30+
| Render success rate | `status` | Derive success as HTTP 2xx. Failures use `0` (no Angular response) or `500` (render exception). |
31+
| Render speed | `ttfb` | Milliseconds from request start until Angular returns a response. Server-side render time, not browser network TTFB. |
32+
| Page completeness | `is_complete` | Checked from rendered HTML. See [is_complete rules](#is_complete-rules). |
33+
| Content type | `content_type` | API resource type from `osf:type` meta (`nodes`, `registrations`, `preprints`, `files`, `users`). Null when not set or not inspected. |
34+
| Bot vs other | `is_bot` | From User-Agent regex in middleware. |
35+
| Crawler identity | `user_agent` | Truncated to 512 characters. |
36+
| Page | `url` | Full public URL (`webUrl` + path). **Query string is stripped** (no `view_only` or other params). |
37+
38+
We do **not** track Search Console index status. “Pages by content type” in SSR means **successful bot responses grouped by `content_type`**, not confirmed Google indexing.
39+
40+
---
41+
42+
## When a metric is sent
43+
44+
Sent for **HTML navigations** handled by the metrics middleware.
45+
46+
**Skipped** (no metric):
47+
48+
- `/assets/*`, `/static/*`
49+
- `/.well-known/*`
50+
- Static file extensions (`.js`, `.css`, `.ico`, `.json`, images, fonts, …)
51+
52+
**Included**:
53+
54+
- Successful renders (any HTTP status Angular returns)
55+
- Missing Angular response → `status: 0`
56+
- Render exception → `status: 500`
57+
58+
---
59+
60+
## When HTML is inspected
61+
62+
HTML is read only when:
63+
64+
- User-Agent matches the bot regex (`SEARCH_BOT` in middleware)
65+
- HTTP status is **200**
66+
67+
If not inspected, `is_complete` stays `false` and `content_type` stays `null` even when the page rendered successfully.
68+
69+
On a bot-only SSR host, every 200 could be inspected; today the code also requires a bot regex match.
70+
71+
Inspection runs **after** the response is sent. It clones the response body only when inspection will run.
72+
73+
---
74+
75+
## is_complete rules
76+
77+
Implemented in `inspectSsrHtml` (`src/server/ssr-html-metrics.ts`).
78+
79+
All must pass:
80+
81+
1. **SSR marker** — `<osf-root>` has `ng-server-context` (page was server-rendered, not a bare CSR shell).
82+
2. **Non-empty root** — inner HTML of `<osf-root>` is not empty after whitespace is removed.
83+
3. **Meta tags or allowlisted path** — either:
84+
- HTML contains `osf-dynamic-meta`, or
85+
- path is on the meta-optional allowlist (search, discover, terms, user, institutions, meetings, collections, provider landing pages, etc.)
86+
87+
---
88+
89+
## content_type
90+
91+
Read from rendered HTML:
92+
93+
```html
94+
<meta name="osf:type" content="nodes" />
95+
```
96+
97+
Set by `MetaTagsService` (`osfType`). Project, registration, preprint, and file pages go through `MetaTagsBuilderService`. Profile pages set `osfType: users` only.
98+
99+
**Populated for:** projects, registrations, preprints, files, users (`/user/:id` and `/profile`).
100+
101+
**Often null for:** institutions, meetings, collections, search, discover, and any page without `osf:type`. That is expected if those types are out of scope for the metric.
102+
103+
---
104+
105+
## Payload and API
106+
107+
**Endpoint**
108+
109+
```
110+
POST {apiDomainUrl}/_/metrics/events/ssr_metrics/
111+
```
112+
113+
**Headers**
114+
115+
- `Accept: application/vnd.api+json;version=2.20`
116+
- `Content-Type: application/vnd.api+json`
117+
- `X-Throttle-Token: {THROTTLE_TOKEN}` when env is set
118+
119+
**Example body**
120+
121+
```json
122+
{
123+
"data": {
124+
"attributes": {
125+
"url": "https://osf.io/abc12/overview",
126+
"ttfb": 842,
127+
"is_bot": true,
128+
"is_complete": true,
129+
"content_type": "nodes",
130+
"status": 200,
131+
"user_agent": "Mozilla/5.0 (compatible; Googlebot/2.1; ...)"
132+
}
133+
}
134+
}
135+
```
136+
137+
If `apiDomainUrl` is missing, the POST is skipped silently.
138+
139+
Failed POSTs are logged with `console.error` in the middleware catch block.
140+
141+
---
142+
143+
## Related docs
144+
145+
- [SSR overview](./ssr.md) — architecture, routes, config
146+
- [Architecture](./arch.md) — file layout

‎docs/ssr.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# SSR (Server-Side Rendering)
2+
3+
## Index
4+
5+
- [Overview](#overview)
6+
- [Production traffic](#production-traffic)
7+
- [Request flow](#request-flow)
8+
- [Key files](#key-files)
9+
- [Render modes](#render-modes)
10+
- [Configuration](#configuration)
11+
- [SEO-related app behavior](#seo-related-app-behavior)
12+
- [Local development](#local-development)
13+
- [Adding or changing SSR routes](#adding-or-changing-ssr-routes)
14+
- [Related docs](#related-docs)
15+
16+
---
17+
18+
## Overview
19+
20+
OSF uses Angular SSR so search-engine crawlers receive fully rendered HTML for public pages (projects, registrations, preprints, discover pages, and similar).
21+
22+
The SSR stack has two layers:
23+
24+
1. **Angular SSR** — renders the app on the server (`app.config.server.ts`, `app.routes.server.ts`).
25+
2. **Express server** — serves static assets, delegates HTML to Angular, and sends render metrics after the response (`src/server.ts`, `src/server/*`).
26+
27+
In production, **only bot traffic** is routed to the SSR build. Human traffic uses the client-only build. See [Production traffic](#production-traffic).
28+
29+
---
30+
31+
## Production traffic
32+
33+
```
34+
Crawler → cloud (bot detection) → SSR Node server → HTML + metrics
35+
Browser → cloud → static/CSR build → SPA shell + client render
36+
```
37+
38+
---
39+
40+
## Request flow
41+
42+
1. Request hits Express (`src/server.ts`).
43+
2. Static files (`/assets`, hashed JS/CSS, etc.) are served from `browser/` when possible.
44+
3. HTML navigations go through `createSsrMetricsMiddleware` (`src/server/ssr-metrics.middleware.ts`).
45+
4. Angular SSR renders the page (`AngularNodeAppEngine.handle`).
46+
5. HTML is sent to the client immediately.
47+
6. In the background (`setImmediate`):
48+
- for bot + HTTP 200: HTML is inspected for completeness and `content_type`
49+
- a metric payload is POSTed to the OSF API
50+
51+
Bot timing is not blocked by steps 6. See [SSR metrics](./ssr-metrics.md).
52+
53+
---
54+
55+
## Key files
56+
57+
| File | Role |
58+
| ---------------------------------------------- | ------------------------------------------------------ |
59+
| `src/server.ts` | Express entry, static files, wires metrics middleware |
60+
| `src/main.server.ts` | Angular server bootstrap |
61+
| `src/app/app.config.server.ts` | Server providers: routes, config, i18n loader |
62+
| `src/app/app.routes.server.ts` | Per-route render mode (Server / Client / Prerender) |
63+
| `src/server/ssr-metrics.middleware.ts` | Render timing, HTML inspection trigger, metric queue |
64+
| `src/server/ssr-html-metrics.ts` | `is_complete` and `content_type` checks |
65+
| `src/server/ssr-metrics.ts` | POST metric payload to API |
66+
| `src/server/ssr-server-config.ts` | Load `config.json` + env for metrics |
67+
| `src/server/static-cache-headers.ts` | Cache headers for static assets |
68+
| `src/app/shared/services/meta-tags.service.ts` | Dynamic SEO meta tags (`osf:type`, `osf-dynamic-meta`) |
69+
70+
---
71+
72+
## Render modes
73+
74+
Defined in `src/app/app.routes.server.ts`:
75+
76+
| Mode | When to use | Bot receives |
77+
| ------------- | ------------------------------------------------------------------------------------ | -------------------------------------------------- |
78+
| **Server** | Public pages that should be indexed (project overview, preprint detail, discover, …) | Full SSR HTML |
79+
| **Client** | Auth, forms, dashboards, moderation, editors | CSR shell; metrics may report `is_complete: false` |
80+
| **Prerender** | Static legal/help pages (`terms-of-use`, `privacy-policy`, …) | Pre-built HTML at build time |
81+
82+
**Rules of thumb**
83+
84+
- Public read-only detail or listing page → **Server**
85+
- Login, submit, edit, settings, “my …” pages → **Client**
86+
- Fixed copy that never changes at runtime → **Prerender**
87+
88+
More specific routes must appear **before** broader patterns (e.g. `preprints/:providerId/:id` before `preprints/:providerId`). Unmatched paths fall through to `**` → **Client**.
89+
90+
---
91+
92+
## Configuration
93+
94+
### `assets/config/config.json`
95+
96+
Copied from `assets/config/template.json` for local dev. Deployed with the build. Used for URLs and third-party keys.
97+
98+
Relevant to SSR / metrics:
99+
100+
| Field | Used by |
101+
| -------------- | ------------------------------------------------------------------- |
102+
| `apiDomainUrl` | SSR API calls (via `OSFConfigService`) and metrics POST target host |
103+
| `webUrl` | Canonical URLs in meta tags; full URL in metric payloads |
104+
105+
### Environment variables (SSR Node process)
106+
107+
| Variable | Used by |
108+
| ---------------- | ----------------------------------------------------------------------- |
109+
| `THROTTLE_TOKEN` | `X-Throttle-Token` on SSR API calls (auth interceptor) and metrics POST |
110+
| `API_DOMAIN_URL` | Fallback if `apiDomainUrl` missing from `config.json` |
111+
| `WEB_URL` | Fallback if `webUrl` missing from `config.json` |
112+
| `PORT` | Express listen port (default `4000`) |
113+
114+
**Throttle token:** SSR page-render API calls read `THROTTLE_TOKEN` from the process environment only (`app.config.server.ts`). It is not taken from `config.json` for auth. Metrics use the same env var via `loadSsrServerConfig`.
115+
116+
### Angular SSR config loading
117+
118+
On the server, `OSFConfigService` does not HTTP-fetch `config.json`. It uses `SSR_CONFIG`, populated at startup from disk in `app.config.server.ts`.
119+
120+
Translations on SSR are loaded from `browser/assets/i18n/en.json` (cached at module load). Browser builds use the HTTP loader as usual.
121+
122+
---
123+
124+
## SEO-related app behavior
125+
126+
### Meta tags
127+
128+
`MetaTagsService` writes dynamic tags with class `osf-dynamic-meta`. Pages also emit `osf:type` (API type: `nodes`, `registrations`, `preprints`, `files`, `users`). Metrics read `osf:type` from the rendered HTML for `content_type`.
129+
130+
Built in `MetaTagsBuilderService` for project, registration, preprint, and file pages. Profile pages call `updateMetaTags` with only `osfType: users` and `mergeDefaults: false`, so they emit a single `osf:type` tag.
131+
132+
---
133+
134+
## Local development
135+
136+
| Command | What it does |
137+
| ----------------------- | --------------------------------------------- |
138+
| `npm start` | CSR dev server (port 4200) |
139+
| `npm run start:ssr` | Dev server with SSR (`dev-ssr` configuration) |
140+
| `npm run build:ssr` | Production SSR build → `dist/osf/` |
141+
| `npm run serve:ssr:osf` | Run built Express server (port 4000) |
142+
143+
Typical local SSR test:
144+
145+
```bash
146+
npm run build:ssr
147+
npm run serve:ssr:osf
148+
```
149+
150+
Set `THROTTLE_TOKEN` in the shell if SSR API calls should bypass throttling locally.
151+
152+
Docker `start:docker` runs the **development** configuration (CSR), not the SSR server. Use `build:ssr` + `serve:ssr:osf` or the Dockerfile `ssr` stage for SSR.
153+
154+
---
155+
156+
## Adding or changing SSR routes
157+
158+
1. Add the route in `app.routes.ts` (browser routes).
159+
2. Add a matching entry in `app.routes.server.ts` with the correct `RenderMode`.
160+
3. If the page should contribute SEO meta, ensure the feature calls `MetaTagsService.updateMetaTags` and sets `osfType` where applicable.
161+
4. For public indexable pages, prefer **Server**. Do not SSR authenticated workflows unless there is a specific SEO need.
162+
163+
---
164+
165+
## Related docs
166+
167+
- [SSR metrics](./ssr-metrics.md) — payload, `is_complete`, dashboards
168+
- [Architecture](./arch.md) — folder layout
169+
- [Docker](./docker.md) — container workflows

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "osf",
3-
"version": "26.19.0",
3+
"version": "26.20.0",
44
"scripts": {
55
"ng": "ng",
66
"analyze-bundle": "ng build --configuration=analyze-bundle && source-map-explorer dist/**/*.js --no-border-checks",

0 commit comments

Comments
 (0)