Skip to content

Conversation

@devsjc
Copy link
Contributor

@devsjc devsjc commented Jan 19, 2026

Contribution Checklist

  • Have you followed the Open Climate Fix Contribution Guidelines?
  • Have you referenced the Issue this PR addresses, where applicable?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added a summary of the changes?
  • Have you written new tests for your changes, where applicable?
  • Have you successfully run make lint with your changes locally?
  • Have you successfully run make test with your changes locally?

Warning

PRs may be closed if all the above boxes are not checked.

Changes in this Pull Request

Speeds up the Timeseries route using DISCTINCT, no CTEs, and scheduled analysis of the forecasts table.
Also speeds up the LatestForecasts route (again!).

Testing on dev I am seeing around ~1ms for GetLatestsForecast and ~10ms for GetForecastAsTimeseries with ~36 million rows in the predicted_generation_values partition and ~10 million in forecasts.

@github-actions
Copy link

github-actions bot commented Jan 19, 2026

Benchmark Results

Benchmark results
?   	github.com/openclimatefix/data-platform/cmd	[no test files]
?   	github.com/openclimatefix/data-platform/internal/gen/ocf/dp	[no test files]
?   	github.com/openclimatefix/data-platform/internal/interceptors	[no test files]
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/dummy	0.005s
{"level":"debug","time":"2026-01-20T15:25:59Z","message":"Completed migrations"}
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkPostgresClient/2400000/GetForecastAsTimeseries-4         	       2	 820466738 ns/op
BenchmarkPostgresClient/2400000/GetForecastAtTimestamp-4          	      72	  16299840 ns/op
BenchmarkPostgresClient/2400000/GetObservationsAsTimeseries-4     	     934	   1213134 ns/op
BenchmarkPostgresClient/2400000/CreateForecast-4                  	      56	  20048555 ns/op
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/postgres	317.468s
?   	github.com/openclimatefix/data-platform/internal/server/postgres/gen	[no test files]
Benchmark vs base branch
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
                                                     │ bench-main.txt │ bench-devsjc-timeseries-speedup.txt │
                                                     │     sec/op     │            sec/op             vs base   │
PostgresClient/6144000/GetForecastAsTimeseries-4         4.826m ± ∞ ¹
PostgresClient/6144000/GetForecastAtTimestamp-4          105.0m ± ∞ ¹
PostgresClient/6144000/GetObservationsAsTimeseries-4     1.307m ± ∞ ¹
PostgresClient/6144000/CreateForecast-4                  2.612m ± ∞ ¹
PostgresClient/2400000/GetForecastAsTimeseries-4                                        820.5m ± ∞ ¹
PostgresClient/2400000/GetForecastAtTimestamp-4                                         16.30m ± ∞ ¹
PostgresClient/2400000/GetObservationsAsTimeseries-4                                    1.213m ± ∞ ¹
PostgresClient/2400000/CreateForecast-4                                                 20.05m ± ∞ ¹
geomean                                                  6.449m                         23.88m        ? ² ³
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
³ ratios must be >0 to compute geomean

@devsjc devsjc merged commit 42ef300 into main Jan 20, 2026
3 checks passed
@devsjc devsjc deleted the devsjc/timeseries-speedup branch January 20, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants