Loading

High memory usage when publishing a large hyper files to Tableau Server

Дата публикации: Apr 13, 2024
Описание

When publishing a large hyper file to Tableau Server from the outside of Tableau Server, such as using Tableau Desktop, tabcmd, or Hyper API, a high memory usage was observed.
For example, when publishing a 30GB hyper file to Tableau Server, a more than 50GB memory usage can be observed.
It causes that the other processes (such as vizportal, backgrounder, vizsqlserver, etc) are terminated by Server Resource Manager for releasing the memory, if the free memory is less than 50GB before the 30GB hyper being published.

As additional information, the following messages can be seen in hyper_0_xxxx.txt log during the hyper file being published toTableau Server.

{"ts":"2022-10-14T06:37:11.571","pid":29240,"tid":"623c","sev":"info","req":"3","sess":"AAAAA2BbBBBBBCC3DDDDEE","ctx":{"sess":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-1:0","req":"YYYYYYYYYYYYYYYYYYYYYYYYYYY","site":"sitename","user":"username","host":"servername","process":"vizportal","workload":"interactive"},"k":"query-begin","v":{"spooling":false,"query-settings-active":false,"query-hash":"abcdefghijklmnopqrstuvwx","query":"VALIDATE DATABASE \"C:/ProgramData/Tableau/Tableau Server/data/tabsvc/temp/vizportal_0.xxxxxxxx/uploadExternalFiles1234567890/extract/hyper_filename.hyper\""}}

 

Cause

This is a function by design.
On Tableau Server 2021.1 and later, Tableau Server starts to validate uploaded extracts (.hyper files) during publishing to see if they are corrupted because they are created on the outside of Tableau Server.
This validating process makes Hyper load the uploaded extract (.hyper file) into the memory for consistency checking, and usually this needs more memory size which is more than the uploaded extract (.hyper file) size, which depends on the compression ratio of the hyper file and the related hyper database files loading.




 
Решение

Option 1:
Use extract refresh to update hyper files on Tableau Server side instead of publishing hyper files from the outside of Tableau Server.

Option 2:
Reduce the size of the extracts (.hyper files).

Option 3:
Add the memory of Tableau Server machine.

Option 4:
Add a dedicated Data Engine (Hyper) node.

​

Дополнительные ресурсы
This validating process does not run to a hyper file which is update by extract refresh on Tableau Server side.
Номер статьи базы знаний

001497154

 
Загрузка
Salesforce Help | Article