Producing png figure using *.vs3 vapor script by "vapor_python"

There is some issue on creating png figures using *.vs3 vapor script. :upside_down_face:

The example vs3 name is “side.vs3” and its result looks like the figure above.
I made it in the vapor of window version(v 3.8.1).

And I would like to produce the figures by each timesteps.

But, my python script seems not to work in “vapor_python” which is installed in the supercomputer.

-------- the example of my python script for producing single snapshot (png format) ----------------
from vapor import session, renderer, dataset,camera, utils
from matplotlib import pyplot as plt

ses1 = session.Session()
ses1.Load(“/h3/home/nimr/dwyi/muklimo/vapor_python/side.vs3”)
cam1 = ses1.GetCamera()
dir1 = cam1.GetDirection()
pos1 = cam1.GetPosition()
up1 = cam1.GetUp()
#ses1.GetCamera().ViewAll()

cam1.SetPosition([pos1[0],pos1[0],pos1[0]])
cam1.SetDirection([dir1[0],dir1[0],dir1[0]])
cam1.SetUp([up1[0],up1[0],up1[0]])
print(“hello”)
ses1.Render(“png/image_1.png”)
plt.savefig(‘png/image_3.png’)


There must be a correct way to save a snapshot after loading the vs3 script.
Please anyone help me. I have struggling with this issue for a long time. :innocent:

FYI, some error messages when I run the python script in linux system. It seems that the version needs to be changed, but I can’t help that. therefore, I would know the way without change the version of GLSL.

Thanks :cowboy_hat_face:

------------------ examples of error messages [1] -------------------------------
Your device appears to either not have hardware graphics support, or not have graphics drivers configured correctly for your device.
Rendering performance may be significantly diminished, or in the case of some renderers, may not function at all.
--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES

Shader “FlowLine.vert” failed to compile
--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES

Shader “FlowLine.frag” failed to compile
Failed to link shader:
Cannot link because shader failed to compile
Resource does not exist and unable to load by name
Failed to initialize renderer sfc_flow
Visualizer (Visualizer_No._0) failed to paint and thus not capturing image.
(vapor_python)

------------------ examples of error messages [2] -------------------------------

Sender: LSF System lsfadmin@pmaru54.maru
Subject: Job 230934: <vapor_n10> in cluster Done

Job <vapor_n10> was submitted from host by user in cluster at Tue Mar 19 05:46:00 2024
Job was executed on host(s) <10*pmaru54>, in queue <normal_p>, as user in cluster at Tue Mar 19 05:46:02 2024
</h3/home/nimr/dwyi> was used as the home directory.
</h3/home/nimr/dwyi/muklimo/vapor_python> was used as the working directory.
Started at Tue Mar 19 05:46:02 2024
Terminated at Tue Mar 19 05:50:53 2024
Results reported at Tue Mar 19 05:50:53 2024

Your job looked like:

LSBATCH: User input

./test.sh

Successfully completed.

Resource usage summary:

CPU time :                                   10.19 sec.
Max Memory :                                 367 MB
Average Memory :                             135.31 MB
Total Requested Memory :                     102400.00 MB
Delta Memory :                               102033.00 MB
Max Swap :                                   -
Max Processes :                              8
Max Threads :                                100
Run time :                                   290 sec.
Turnaround time :                            293 sec.

The output (if any) follows:

[GLContextProviderNvidia.cpp:20] Could not load EGL
[GLContextProviderEGL.cpp:27] Could not load EGL
[GLContextProvider.cpp:28] Could not get an OpenGL context from the display manager. Is one running?
[GLContextProvider.cpp:29] Falling back to software rendering
Vapor 3.9.0
Python 3.9.18 (/h3/home/nimr/dwyi/miniforge3/envs/vapor_python)
OpenGL 3.3 (Core Profile) Mesa 21.2.5
hihi
Dataset: ‘all.202310300000to202311090800.nc’
[RenderManager.cpp:222] Paint Failed

Hi @muteldw, could you share your entire script? The one you provided prints “hello” and the output prints “hihi” so there’s a discrepency here. Could you also share a single timestep of your simulation so we can try to reproduce this?

Also, are you running VAPOR’s gui on the same machine that you’re running the python_api on?

Hi, @pearse
Many thanks for the response.

It is entire script and actually the message of “hihi” is correct. I just modified while I was leaving a question. Sorry for the confusion. :melting_face:

I would upload entire script with files if it is possible. however, there is no uploading option in this website. so, I am leaving downloadable link for you. :desktop_computer:

And I am worried that I don’t understand of your question. I am currently running VAPOR in Window (the sample figure is also produced in window system) and the version of “vapor_python” is installed in LINUX system for the automation for producing figures (it is not working well though… TT).

I leave the sample files (zip format) at the following link. it is my personal blogs. :laughing:
https://cafe.naver.com/greatme/11
(you can find the attachment, “sample of single timestep for vapor.zip” :floppy_disk:)

I reorganized files for you.

  • LIDAR_Gohung_202402051800.nc.vapor.nc ← original file (one timestep is extracted because of file size issue)

  • for_WINDOW_airplane_view.vs3 ← you can see the similar figure with same camera view in window)

  • for_LINUX_airplane_view.vs3 ← working in test.py (some of lines are modified fo the correct path)

The working process of producing figure in the super-computer is,

  1. run.sh → 2. test.sh → 3. test.py

mp276812.err and mp276812.out are the log files after finishing the process.

I hope it is useful for helping me. Thanks for your consideration. :smiley:

Thanks - Could you email the .zip as an attachment to vapor@ucar.edu? Sorry, I should have asked that initially. Cool blog by the way, google translate did a great job for it.

And I am worried that I don’t understand of your question. I am currently running VAPOR in Window (the sample figure is also produced in window system) and the version of “vapor_python” is installed in LINUX system for the automation for producing figures (it is not working well though… TT).

I wasn’t sure what you meant by “running VAPOR in Window” because VAPOR’s GUI is run in a window. The Python scripting API can also direct its image output to new windows on the desktop. Adding to all of this, VAPOR also supports the Windows operating system.

But I think I know which windows we are talking about here: You’re running VAPOR’s desktop GUI to configure a session on Linux. You are then using the Python API on the same system. Is that correct?

Thanks. I am just sending an attachment to the email.

The “running VAPOR in Window” means that I get a *.vs3 by the vapor of window version.
Then, I believe that I can get the png file of captured image after running *.vs3 at the Linux system. Because the current Linux system of the super-computer in my company does not support GUI with the limited computing resources for the users by strict government policy. :cold_sweat:

and the Python API (vapor python with conda) is installed in the Linux system that I can access through MobaXterm (SSH client for window).

The purpose of this work is to build an automation system to produce images in real-time in the Linux system. Therefore, the “vapor_python (without GUI)” was become a good solution for me, not a “vapor with linux version (with GUI)”. :smile_cat:

[example figure. it is so many… :melting_face:]

Sorry for the insufficient explanation. I’m still not familiar with the term. :smiling_face_with_tear:

Thanks

@muteldw - Does your compute node have a graphics card? I believe this is the issue. Some of VAPOR’s renderers support software rendering which can be done without a dedicated graphics processor. The Flow renderer requires this though.

If not, Is there an alternative cluster you can execute upon, that has a graphics card?

@pearse - Although the computing node has a graphics card, the GUI cannot be launched from the login server due to national computer security policy. So, I think it would be possible to draw a picture with a python script in VAPOR_PYTHON. Isn’t it?

And the current computing node (LENOVO SD650, 51 PF) has a graphics card.

I would be very grateful if you could suggest an example script that can produce and save pictures using VAPOR_PYTHON despite the error message below.

--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES

Shader “FlowLine.vert” failed to compile
--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES

Shader “FlowLine.frag” failed to compile

@muteldw Sorry for the late reply, I was on PTO for spring break last week.

This is tricky because I can’t reproduce those errors, and I can produce imagery with the python_api from the session and data file you provided. I think this is a system issue.

Since your system does have a graphics card, can you share what graphics card is installed? Also, do you know if the system has ever been able to run another OpenGL based application?

I’ll discuss this with my team tomorrow.

1 Like

@muteldw Just following up. We do think that this is a system issue. I’m able to run your script locally on my laptop. If you know what graphics card is in use, and whether your cluster has run another OpenGL application before, we may be able to help troubleshoot.

1 Like

Hi Pearse!

Thank you very much for the response.
And sorry for my late reply.

I am sure that it will be able to work well on a personal laptop or computer.
However, there some technical issue on the super-computer as a government resource.
such as limitation of usage of CPUs, memory, etc…

My team would setup a stable system with VAPOR that can produce figures automatically to support for the real-time monitoring system.

And yes, you are right. it is a system issue. I know only that our super-computer system is built by LENOVO SD650 (CPU: 612,864, Theoretical performance: 51 PF, Memory: 2,064 TB). I hope this information can be useful to you.

I don’t need to work and modify any script on Linux system. I just want to run the vapor script (*.vs3) by “vapor_python” and get some multiple images with timesteps. Because I can make and save the perfect script on window version of vapor.

Please let’s me know some example for vapor_python. I already installed vapor_python based on a conda (miniforge3).

--------- my example -------------------------------------------------
------- in the shell --------
module purge
source ${HOME}/miniforge3/etc/profile.d/conda.sh
conda activate vapor_python

------- in the vapor_python script --------
from vapor import session, renderer, dataset,camera, utils
from matplotlib import pyplot as plt

ses1 = session.Session()
ses1.Load(“/h3/home/nimr/dwyi/muklimo/vapor_python/side.vs3”)
cam1 = ses1.GetCamera()
dir1 = cam1.GetDirection()
pos1 = cam1.GetPosition()
up1 = cam1.GetUp()
#ses1.GetCamera().ViewAll()

cam1.SetPosition([pos1[0],pos1[0],pos1[0]])
cam1.SetDirection([dir1[0],dir1[0],dir1[0]])
cam1.SetUp([up1[0],up1[0],up1[0]])
print(“hi”)
ses1.Render(“png/image_1.png”)
plt.savefig(‘png/image_3.png’)

Hi @muteldw, no problem. I’m a little bit unclear about your question. Is that script not working for you? I was able to get a similar script to work on my local machine a few weeks ago, which I’ve copied below.

#!/usr/bin/env python3
from vapor import session, renderer, dataset,camera, utils
from matplotlib import pyplot as plt

# Acquire the camera settings from an initial session file
ses1 = session.Session()
#ses1.Load("/h3/home/nimr/dwyi/muklimo/vapor_python/tmp/for_LINUX_airplane_view.vs3")
ses1.Load("/Users/vapor/Downloads/muteldw/for_LINUX_airplane_view.vs3")
cam1 = ses1.GetCamera()
dir1 = cam1.GetDirection()
pos1 = cam1.GetPosition()
up1 = cam1.GetUp()

cam1.SetPosition([pos1[0],pos1[0],pos1[0]])
cam1.SetDirection([dir1[0],dir1[0],dir1[0]])
cam1.SetUp([up1[0],up1[0],up1[0]])
print("hihi")
ses1.Render("image_1.png")
plt.savefig('image_3.png')
1 Like

No, it isn’t working with the same script as you mentioned.

It seems to be required lower version of GLSL belows 4.10. I wonder it would be work if the lower version of GLSL is installed.

It would be the similar question and answer as we shared on Mar 25.
I attached the logs after running the script by super-computer.

Thanks…

[log1: mp693355.err]
Your device appears to either not have hardware graphics support, or not have graphics drivers configured correctly for your device.
Rendering performance may be significantly diminished, or in the case of some renderers, may not function at all.
--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES
Shader “FlowLine.vert” failed to compile

--------------- Shader Compilation Failed ---------------
0:1(10): error: GLSL 4.10 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.50, 3.30, 1.00 ES, 3.00 ES, and 3.10 ES

Shader “FlowLine.frag” failed to compile
Failed to link shader:
Cannot link because shader failed to compile
Resource does not exist and unable to load by name
Failed to initialize renderer sfc_flow
Visualizer (Visualizer_No._0) failed to paint and thus not capturing image.

[log2: mp693355.out]
Sender: LSF System lsfadmin@pmaru60.maru
Subject: Job 693355: <vapor_n10> in cluster Done

Job <vapor_n10> was submitted from host by user in cluster at Tue Apr 23 22:50:12 2024
Job was executed on host(s) <10*pmaru60>, in queue <normal_p>, as user in cluster at Tue Apr 23 22:50:13 2024
</h3/home/nimr/dwyi> was used as the home directory.
</h3/home/nimr/dwyi/muklimo/vapor_python> was used as the working directory.
Started at Tue Apr 23 22:50:13 2024
Terminated at Tue Apr 23 22:52:27 2024
Results reported at Tue Apr 23 22:52:27 2024

Your job looked like:

'# LSBATCH: User input
./test.sh

Successfully completed.

Resource usage summary:

CPU time :                                   14.16 sec.
Max Memory :                                 367 MB
Average Memory :                             205.36 MB
Total Requested Memory :                     102400.00 MB
Delta Memory :                               102033.00 MB
Max Swap :                                   -
Max Processes :                              5
Max Threads :                                100
Run time :                                   133 sec.
Turnaround time :                            135 sec.

The output (if any) follows:

[GLContextProviderNvidia.cpp:20] Could not load EGL
[GLContextProviderEGL.cpp:27] Could not load EGL
[GLContextProvider.cpp:28] Could not get an OpenGL context from the display manager. Is one running?
[GLContextProvider.cpp:29] Falling back to software rendering
Vapor 3.9.0
Python 3.9.18 (/h3/home/nimr/dwyi/miniforge3/envs/vapor_python)
OpenGL 3.3 (Core Profile) Mesa 21.2.5
hihi
Dataset: ‘all.202310300000to202311090800.nc’
[RenderManager.cpp:222] Paint Failed

PS:

Read file <mp693355.err> for stderr output of this job.

===

The error is reporting that your graphics drivers are not installed, or are extremely old.

Your device appears to either not have hardware graphics support, or not have graphics drivers configured correctly for your device.

What operating system are you using? I can help you find instructions on how to configure it.

1 Like

The super-computer in our laboratory is using CentOS Linux (release 8.3.2011). If you give some instructions, I will ask to the staffs of the super-computing centers if the instructions can be applied with government policy.

Thank you so much.

Ok, the following link steps through the process:

I’m surprised that your supercomputer has NVIDIA graphics cards but graphics drivers have not been configured. There is no point in having graphics hardware on a system without drivers. This should be an urgent issue for a supercomputer administrator.

Also, you mentioned the following:

My team would setup a stable system with VAPOR that can produce figures automatically to support for the real-time monitoring system.

Is CentOS8 also the operating system for this system? I would encourage your team to configure this system while the HPC administrators sort out their graphics driver.

@muteldw, do you happen to know if your systems are installed with drivers specific for CUDA? These drivers may not be suited for rendering, but for computation instead.