Events listing with UI is sub-optimal and might need improvement


A page <cloudify-ui>/console/page/logs fetches all the records from events and logs tables, combining them with four or five others.

We ( and ) tried optimizing it but faced with:

  • issues with pagination sorting/limiting/offsetting: there is no easy way to limit the records being fetched at the very beginning of building a query since it is union-ed data that is later sort-ed/limit-ed/ofset-ed,

  • issues with pagination: every time the UI request is sent, all the combined events and logs records, joined with four or five another tables are being count-ed

All of the above make some ideas described in obsolete. We just cannot quickly count over 15 millions of records over five to seven SQL tables.

So two ideas popped up:

  • get rid of this view from UI completely (who needs to browse all the logs, from all the deployments, page by page, being able at the same time to jump exactly to page fifteen hundreds?)

  • add some kind of counter-cache over those large tables that would allow us to get rid of count(*) on every GET /events request

  • change pagination on the logs page for one based no on limit/offset but limit/greater-than, i.e. select 50 elements with timestamp greater than this one, sorted by timestamp


Mateusz Neumann
February 5, 2021, 12:31 PM

Look at how pagination works at Google’s search service, or a pi-hole.




Mateusz Neumann


Target Version


Premium Only


Release Notes