Kernelshark v2 all-black plots area
Heya, me again! :-)
So, I'm not playing with trace-cmd only, I'm also giving a try to kernelshark-v2.
I can build it fine, and it even starts, but the plots area looks almost all black. The events and the tasks occurring on the "last CPU" (i.e., on CPU7 if I have 8 CPUs) are more or less fine, it seems (although still hard to work with, given the background).
Actually, it's the last plot that "works". In fact, if I add a task plot, I do see what that one is supposed to show, but the events of CPU7 disappear.
Others are there, because the marker moves if I click on the entries in the trace area... but they're kind of behind the black cover.
A screenshot can be found here:
http://xenbits.xen.org/people/dariof/kernelshark-v2-black.png
Am I the only one? Can it be related to my configuration or build environment?
I'm happy to help with more info or try some debugging, if you think it could be useful.
Thanks and Regards
Signature · 6 lines
Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
Ciao Dario,
There is a bug in one of the patches from the last patch-set I sent. This bug was reported by Tzvetomir. My guess is that you are hitting the same issue. If this is the case you must also have an error message saying "Unable to find FreeSans font".
You can manually do the fix suggested by Tzvetomir (see his reply to [PATCH 15/24] kernel-shark: Update KsDualMarker and KsGLWidget) or just get a corrected (rebased) version of the code from
https://github.com/yordan-karadzhov/kernel-shark-v2.beta
Please let me know if this fixes your issue.
Thank you very much for testing and reporting!
Best
Yordan
Quoted 28 lines · Dario Faggioli
Heya, me again! :-)
So, I'm not playing with trace-cmd only, I'm also giving a try to kernelshark-v2.
I can build it fine, and it even starts, but the plots area looks almost all black. The events and the tasks occurring on the "last CPU" (i.e., on CPU7 if I have 8 CPUs) are more or less fine, it seems (although still hard to work with, given the background).
Actually, it's the last plot that "works". In fact, if I add a task plot, I do see what that one is supposed to show, but the events of CPU7 disappear.
Others are there, because the marker moves if I click on the entries in the trace area... but they're kind of behind the black cover.
A screenshot can be found here:
http://xenbits.xen.org/people/dariof/kernelshark-v2-black.pngAm I the only one? Can it be related to my configuration or build environment?
I'm happy to help with more info or try some debugging, if you think it could be useful.
Thanks and Regards
Quoted 2 lines · Yordan Karadzhov (VMware)
Ciao Dario,
Ciao! :-)
Quoted 6 lines
There is a bug in one of the patches from the last patch-set I sent. This bug was reported by Tzvetomir. My guess is that you are hitting the
same issue. If this is the case you must also have an error message saying "Unable to find FreeSans font".
Yep, I was seeing that too. And, TBH, for a while I installed font packages like mad, thinking it was related to something like that. :-P
Quoted 6 lines
You can manually do the fix suggested by Tzvetomir (see his reply to [PATCH 15/24] kernel-shark: Update KsDualMarker and KsGLWidget) or just
get a corrected (rebased) version of the code from
https://github.com/yordan-karadzhov/kernel-shark-v2.beta
Ah, great! I updated (rebase included), and now plots are back.
Quoted 2 lines
Please let me know if this fixes your issue.
Indeed it does, thanks a lot again.
Now, if I try "KVM Combo Plot":
- I still see super weird results if I use the KVM clock from last patch in Tzvetomir series (but I guess I'll report it there).
- If I use PTP, that kind of works.
If I trace a sleep 1 on the host, the result is like this:
http://xenbits.xen.org/people/dariof/kernelshark-v2-ptp.png
So, there is ~ 1 sec of data from the guest, coming from trace-cmd agent. The host, however, went on and traced a little bit longer, even after closing the connection. It's not too terrible, I'm just curious whether this is known or it's something weird and it's only me seeing it.
Evaluating the correctness and the accuracy of the synchronization is a little bit of a challenge, especially for me that I'm not used to it. And, at least in part, because I still don't have this change in the kernel I'm using right now, so some kvm_entry trace points are not recorded, which means it's tricky pairing VMEntries and VMExits:
https://lore.kernel.org/lkml/160873470698.11652.13483635328769030605.stgit@Wayrath/
It should come with 5.11, but I think I'll quickly apply it to the 5.10 that I'm running, to try to understand better hat's going on.
Thanks and Regards
Signature · 6 lines
Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
Hi Dario,
Quoted 5 lines · Dario Faggioli
Indeed it does, thanks a lot again.
Now, if I try "KVM Combo Plot":
- I still see super weird results if I use the KVM clock from last patch in Tzvetomir series (but I guess I'll report it there).
Can you share a screenshot of the one that looks weird. And please do some deeper zooming because it is very hard to judge about the quality of the sync from such a broad time window.
Quoted 5 lines
- If I use PTP, that kind of works.
If I trace a
sleep 1on the host, the result is like this:
Is this a PTP synchronization?
Quoted 6 lines
So, there is ~ 1 sec of data from the guest, coming from trace-cmd agent. The host, however, went on and traced a little bit longer, even after closing the connection. It's not too terrible, I'm just curious whether this is known or it's something weird and it's only me seeing it.
I think this is normal.
But anyway Ceco (Tzvetomir) is the right person to answer questions about the timestamp synchronization. Unfortunately he is on vacation till the end of the week.
Thanks a lot!
Yordan
Quoted 11 lines
Evaluating the correctness and the accuracy of the synchronization is a little bit of a challenge, especially for me that I'm not used to it. And, at least in part, because I still don't have this change in the kernel I'm using right now, so some
kvm_entrytrace points are not recorded, which means it's tricky pairing VMEntries and VMExits:https://lore.kernel.org/lkml/160873470698.11652.13483635328769030605.stgit@Wayrath/
It should come with 5.11, but I think I'll quickly apply it to the 5.10 that I'm running, to try to understand better hat's going on.
Quoted 12 lines · Yordan Karadzhov (VMware)
Quoted 5 lines · Dario Faggioli
Indeed it does, thanks a lot again.
Now, if I try "KVM Combo Plot":
- I still see super weird results if I use the KVM clock from last patch in Tzvetomir series (but I guess I'll report it there).Can you share a screenshot of the one that looks weird. And please do some deeper zooming because it is very hard to judge about the quality
of the sync from such a broad time window.
Sure, let me try.
So, while tracing, I see this, so I think I'm in fact using the KVM clock (ignore the "(null)" thing, we're dealing with it in another thread):
Negotiated kvm time sync protocol with guest (null)
When start KernelShark 2 on trace.dat (the host trace), I see this on the terminal:
./kernel-shark-v2.beta.git/bin/kernelshark trace.dat
loading plugin "sched_events" from /home/dario/Sources/tracing/kernel-shark-v2.beta.git/lib/plugin-sched_events.so loading plugin "missed_events" from /home/dario/Sources/tracing/kernel-shark-v2.beta.git/lib/plugin-missed_events.so loading plugin "event_field_plot" from /home/dario/Sources/tracing/kernel-shark-v2.beta.git/lib/plugin-event_field_plot.so loading plugin "latency_plot" from /home/dario/Sources/tracing/kernel-shark-v2.beta.git/lib/plugin-latency_plot.so loading plugin "kvm_combo" from /home/dario/Sources/tracing/kernel-shark-v2.beta.git/lib/plugin-kvm_combo.so --> KVM combos init menu
--> latency_plot init menuplugin_latency_add_menu 0x100aea0
--> event_field init menuplugin_efp_add_menu 0xf51510
--> sched init menu
Loading "trace.dat"
version = 6
And this is how the GUI looks:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host.png
Then, I append the trace-(null).dat file (the guest trace). I do see this on the terminal:
overriding event (1) ftrace:function with new print handler
overriding event (10) ftrace:funcgraph_exit with new print handler
overriding event (11) ftrace:funcgraph_entry with new print handler
<-- sched close 0
<-- missed_events close 0
<-- KVM combos close 0
--> KVM combos init 0
--> missed_events init 0
--> sched init 0
--> KVM combos init 1
plugin "kvm_combo" failed to initialize on stream /home/dario/Sources/tracing/trace-(null).dat --> missed_events init 1
--> sched init 1
And the GUI looks like this:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host_guest.png
The guest has 1 vCPU and is pinned to host pCPU 1. There are two yes task running inside it. On the host, there are two yes task too, also pinned to vCPU 1, so that there's some resource contention.
This is the very same configuration of the other experiment, for which I showed the screenshot in the other email, when using PTP.
There, things were kind of ok. Here, it seems that the host events are gone, but if I zoom a lot at the very beginning of the trace, I do see them again:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host_guest-zoom.png
Point is that the traces are not in sync. Actually, it looks to me that the timestamps of the guest traces are the most weird part, as they look rather big, not really consistent among each other and some are even negative!
IAC, the events from the two traces are shown as two completely disjoint sets:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host_guest-events.png
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host_guest-events-zoom.png.png
If I try a KVM-Combo, this is how it looks (not really surprisingly):
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host_guest-combo.png
Note that if I just open the guest trace in a new session, it looks fine:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-guest.png
I'm uploading the two trace files here, in case it could be useful:
http://xenbits.xen.org/people/dariof/ks2-debug/
Quoted 8 lines
Quoted 5 lines
- If I use PTP, that kind of works.
If I trace a
sleep 1on the host, the result is like this:Is this a PTP synchronization?
It is, yes.
Quoted 11 lines
Quoted 8 lines
So, there is ~ 1 sec of data from the guest, coming from trace-cmd agent. The host, however, went on and traced a little bit longer, even
after closing the connection. It's not too terrible, I'm just curious
whether this is known or it's something weird and it's only me seeing
it.I think this is normal.
Right.
Quoted 4 lines
But anyway Ceco (Tzvetomir) is the right person to answer questions about the timestamp synchronization. Unfortunately he is on vacation till the end of the week.
Sure, no problem!
Thanks and Regards
Signature · 6 lines
Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
Hi Dario,
Quoted 6 lines · Dario Faggioli
And this is how the GUI looks:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host.png
Then, I append the trace-(null).dat file (the guest trace). I do see this on the terminal:
I see what you mean. In this case the host and the guest are not synchronized at all. I guess something went wrong during the recording. Also the fact that the name of the guest VM is "(null)" looks very suspicious. I guess this is not the real name of the VM you are using?
Steven, do you have any idea what can cause such problem?
Thanks!
Yordan
Quoted 5 lines · Yordan Karadzhov (VMware)
Quoted 3 lines · Dario Faggioli
Then, I append the trace-(null).dat file (the guest trace). I do see
this on the terminal:
Quoted 7 lines
I see what you mean. In this case the host and the guest are not synchronized at all. I guess something went wrong during the recording.
Also the fact that the name of the guest VM is "(null)" looks very suspicious. I guess this is not the real name of the VM you are using?
Well, it is weird, but I think Tzvetomir figured it out. Or at least, it's probably an issue while parsing the qemu command line for trying to figure out the name of the guest.
It's indeed strange and annoying, but I don't think it necessarily means the trace is not valid.
Also, it sure can be something like that. But if I just open the guest trace alone (or I check the records with trace-cmd report), it seems fine and the timestamps are not crazy.
The screenshot of me doing that is this one:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-guest.png
Quoted 2 lines
Steven, do you have any idea what can cause such problem?
The trace is here:
http://xenbits.xen.org/people/dariof/ks2-debug/trace-(null).dat
With trace-cmd dump --options, I see this:
sudo trace-cmd dump --options trace-\(null\).dat
[Option TRACECLOCK, 0 bytes]
[Option UNAME, 47 bytes]
Linux tumbleweed-jeos 5.10.12-1-default x86_64
[Option VERSION, 47 bytes]
2.9.1 1b6bba6c0682975377477d5eabfd541c060aea03
[Option TRACEID, 8 bytes]0xD663C16F20E1B52E
[Option TimeShift, 68 bytes]0xD8F36FC634D63137 [peer's trace id]0x0 [peer's protocol flags]0x1 [peer's CPU count]
2 [samples count for CPU 0]
71075647069541 281474976710656 8311226241638 [offset * scaling @ time] 71075647069541 281474976710656 8315062444376 [offset * scaling @ time]
Thanks and Regards
Signature · 6 lines
Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
Quoted 10 lines · On Wed, 10 Feb 2021 17:47:16 +0200 "Yordan Karadzhov (VMware)"…
Hi Dario,
Quoted 6 lines · Dario Faggioli
And this is how the GUI looks:
http://xenbits.xen.org/people/dariof/ks2-debug/kernelshark-v2-host.png
Then, I append the trace-(null).dat file (the guest trace). I do see this on the terminal:
Quoted 6 lines
I see what you mean. In this case the host and the guest are not synchronized at all. I guess something went wrong during the recording. Also the fact that the name of the guest VM is "(null)" looks very suspicious. I guess this is not the real name of the VM you are using?
Steven, do you have any idea what can cause such problem?
There's currently a lot of bugs that I'm trying to wring out.
Yes, I see the same thing, but haven't had the time to investigate it further. I have to push the few fixes I have in queue out after some tests, and then I'll look into this. My main concern was that the "listen" functionality broke, which is a regression!
-- Steve
On Tue, Feb 09, 2021 at 03:06:14PM +0200, Yordan Karadzhov (VMware) wrote:
Hi Yordan!
I have the same issue of an all-black plots area.
Quoted 6 lines
Ciao Dario,
There is a bug in one of the patches from the last patch-set I sent. This bug was reported by Tzvetomir. My guess is that you are hitting the same issue. If this is the case you must also have an error message saying "Unable to find FreeSans font".
I have the same error message. I tried installing the gnu-free-fonts package on Arch, but it does not get rid of the error.
Quoted 5 lines
You can manually do the fix suggested by Tzvetomir (see his reply to [PATCH 15/24] kernel-shark: Update KsDualMarker and KsGLWidget) or just get a corrected (rebased) version of the code from
https://github.com/yordan-karadzhov/kernel-shark-v2.beta
I tried installing from your github repo, but the same error still shows. I have all other dependencies installed. FreeSans is the only complaint from the cmake .. command. Is there another potential fix I could try?
Quoted 6 lines
Please let me know if this fixes your issue.
Thank you very much for testing and reporting!Best
Yordan
Thank you!
Cheers,
Solomon
Hi Solomon,
There are few things you can do.
The way KernelShark build finds the font file is by running the following command in a shell:
fc-list FreeSans |grep FreeSans.ttf | cut -d':' -f 1 -z
So run the same command in a terminal and try to figure out why it fails.
If it fails because the 'FreeSans.ttf' does not exist, you can manually provide a path to some other font file to be used.
cd kernel-shark/build
./cmake_clean.sh
cmake -DTT_FONT_FILE=/path/to/some/font/YourFont.ttf ..
Please let me know if this works.
Thanks!
Yordan
Quoted 36 lines · Solomon Tan
On Tue, Feb 09, 2021 at 03:06:14PM +0200, Yordan Karadzhov (VMware) wrote:
Hi Yordan!
I have the same issue of an all-black plots area.
Quoted 6 lines
Ciao Dario,
There is a bug in one of the patches from the last patch-set I sent. This bug was reported by Tzvetomir. My guess is that you are hitting the same issue. If this is the case you must also have an error message saying "Unable to find FreeSans font".
I have the same error message. I tried installing the
gnu-free-fontspackage on Arch, but it does not get rid of the error.Quoted 5 lines
You can manually do the fix suggested by Tzvetomir (see his reply to [PATCH 15/24] kernel-shark: Update KsDualMarker and KsGLWidget) or just get a corrected (rebased) version of the code from
https://github.com/yordan-karadzhov/kernel-shark-v2.betaI tried installing from your github repo, but the same error still shows. I have all other dependencies installed. FreeSans is the only complaint from the cmake .. command. Is there another potential fix I could try?
Quoted 6 lines
Please let me know if this fixes your issue.
Thank you very much for testing and reporting!Best
YordanThank you!
Cheers,
Solomon
Quoted 9 lines · Yordan Karadzhov
Hi Solomon,
There are few things you can do.
The way KernelShark build finds the font file is by running the following command in a shell:fc-list FreeSans |grep FreeSans.ttf | cut -d':' -f 1 -z
So run the same command in a terminal and try to figure out why it fails.
If it fails because the 'FreeSans.ttf' does not exist, you can manually provide a path to some other font file to be used.
It failed because FreeSans was installed in my system as otf instead of ttf hahaha.
Quoted 4 lines
cd kernel-shark/build
./cmake_clean.sh
cmake -DTT_FONT_FILE=/path/to/some/font/YourFont.ttf ..
Thank you! This command worked. I just specified the path to the otf file instead of the ttf file, and it worked like a charm. kernel-shark is installed and working well!
I actually faced another issue as well - which was that the compilation appeared to be successful but kernelshark was not made at all because FreeSans was missing. So the black plot I got was from the one I installed from the package manager. But now I have kernelshark installed from your github so all is good now :).
I would like to propose some changes to the CMakefile in light of my experience. I will send them in reply to this email shortly. For your consideration, please :).
Thank you!
Cheers,
Solomon
Quoted 43 lines
Please let me know if this works.
Thanks!
YordanQuoted 36 lines · Solomon Tan
On Tue, Feb 09, 2021 at 03:06:14PM +0200, Yordan Karadzhov (VMware) wrote:
Hi Yordan!
I have the same issue of an all-black plots area.
Quoted 6 lines
Ciao Dario,
There is a bug in one of the patches from the last patch-set I sent. This bug was reported by Tzvetomir. My guess is that you are hitting the same issue. If this is the case you must also have an error message saying "Unable to find FreeSans font".
I have the same error message. I tried installing the
gnu-free-fontspackage on Arch, but it does not get rid of the error.Quoted 5 lines
You can manually do the fix suggested by Tzvetomir (see his reply to [PATCH 15/24] kernel-shark: Update KsDualMarker and KsGLWidget) or just get a corrected (rebased) version of the code from
https://github.com/yordan-karadzhov/kernel-shark-v2.betaI tried installing from your github repo, but the same error still shows. I have all other dependencies installed. FreeSans is the only complaint from the cmake .. command. Is there another potential fix I could try?
Quoted 6 lines
Please let me know if this fixes your issue.
Thank you very much for testing and reporting!Best
YordanThank you!
Cheers,
Solomon
PATCH 1/2 kernel-shark: Make FreeSans a compulsory component
This patch is proposed for two reasons.
Firstly, with reference to
https://lore.kernel.org/linux-trace-devel/Yl9WBztqKGD0PpmJ@ArchDesktop/
when the CMake does not detect FreeSans, it should cease generating the Makefile since kernelshark requires FreeSans to output its graph properly.
Secondly, when the make instructions on the README are followed, even when FreeSans is not detected, make continues to compile and make install (or install_gui) would install without indicating an error. This gives the impression of a successful install, but kernelshark will be missing.
Therefore, making CMake quit with error when FreeSans is not installed solves these two issues.
--- CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
CMakeLists.txt +1 -1
@@ -69,7 +69,7 @@ if (TT_FONT_FILE) else (TT_FONT_FILE) - message(WARNING "\nCould not find font ${KS_FONT}!\n") + message(FATAL_ERROR "\nCould not find font ${KS_FONT}!\n") endif (TT_FONT_FILE)
Signature · 1 line
2.36.0
PATCH 2/2 kernel-shark: Allow FreeSans otf in addition to ttf
FreeSans comes as otf rather than ttf in some cases. An example is the gnu-free-fonts on Arch. With this patch, both FreeSans.otf and FreeSans.ttf will be accepted to make kernelshark.
--- CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
CMakeLists.txt +1 -1
@@ -59,7 +59,7 @@ find_package(GLUT) set(KS_FONT FreeSans) if (NOT TT_FONT_FILE) - execute_process(COMMAND bash "-c" "fc-list '${KS_FONT}' |grep ${KS_FONT}.ttf | cut -d':' -f 1 -z" + execute_process(COMMAND bash "-c" "fc-list '${KS_FONT}' |grep -E ${KS_FONT}'(\.otf|\.ttf)' | cut -d':' -f 1 -z" OUTPUT_VARIABLE TT_FONT_FILE) endif (NOT TT_FONT_FILE)
Signature · 1 line
2.36.0
Re: [PATCH 2/2] kernel-shark: Allow FreeSans otf in addition to ttf
Hi Solomon,
I am taking this one.
Thanks!
Yordan
Quoted 22 lines · Solomon Tan
FreeSans comes as otf rather than ttf in some cases. An example is the gnu-free-fonts on Arch. With this patch, both FreeSans.otf and FreeSans.ttf will be accepted to make kernelshark.
Signed-off-by Solomon Tan <[email protected]>--- CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)CMakeLists.txt +1
-1@@ -59,7 +59,7 @@ find_package(GLUT) set(KS_FONT FreeSans) if (NOT TT_FONT_FILE) - execute_process(COMMAND bash "-c" "fc-list '${KS_FONT}' |grep ${KS_FONT}.ttf | cut -d':' -f 1 -z" + execute_process(COMMAND bash "-c" "fc-list '${KS_FONT}' |grep -E ${KS_FONT}'(\.otf|\.ttf)' | cut -d':' -f 1 -z" OUTPUT_VARIABLE TT_FONT_FILE) endif (NOT TT_FONT_FILE)
Re: [PATCH 1/2] kernel-shark: Make FreeSans a compulsory component
In reply to Solomon TanQuoted 27 lines · Solomon Tan
This patch is proposed for two reasons.
Firstly, with reference to
https://lore.kernel.org/linux-trace-devel/Yl9WBztqKGD0PpmJ@ArchDesktop/
when the CMake does not detect FreeSans, it should cease generating theMakefilesince kernelshark requires FreeSans to output its graph properly.Secondly, when the make instructions on the README are followed, even when FreeSans is not detected,
makecontinues to compile andmake install(orinstall_gui) would install without indicating an error. This gives the impression of a successful install, butkernelsharkwill be missing.Therefore, making CMake quit with error when FreeSans is not installed solves these two issues.
Signed-off-by Solomon Tan <[email protected]>--- CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)CMakeLists.txt +0
-0
CMakeLists.txt if (TT_FONT_FILE) · line 72
Quoted 5 lines
CMakeLists.txt +1
-1@@ -69,7 +69,7 @@ if (TT_FONT_FILE) else (TT_FONT_FILE) - message(WARNING "\nCould not find font ${KS_FONT}!\n") + message(FATAL_ERROR "\nCould not find font ${KS_FONT}!\n")
Hi Solomon,
Having this message as warning instead of a fatal error is done on purpose. We have other projects, that are using the kernelshark library (libkshark) but do not need the kernelshark GUI. So the library have to be able to build even if the third party packages needed by the GUI are not installed.
On the other hand I understand your argument that this can be confusing for someone who wants to build the GUI and do not care about the library. My suggestion is to keep the warning, but to make the message more informative, so that the user will be aware that the GUI is not going build.
And please send your patch-sets as new email threads.
Thanks!
Yordan
Quoted 3 lines
CMakeLists.txt +0
-0endif (TT_FONT_FILE)