View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0014729ParaView(No Category)public2014-05-14 10:422015-01-11 08:18
ReporterVadim SANDLER 
Assigned ToJoachim Pouderoux 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version4.1 
Target VersionFixed in Version4.2 
Summary0014729: It is impossible use Unicode symbols in the "Text" source
DescriptionIt is impossible use Unicode symbols in the "Text" source from the python or directly from the GUI:

Example 1:

Text1 = Text()
Text1.Text = 'Diff\xe9rence'
RenderView1 = GetRenderView()
DataRepresentation1 = Show()
Render()

Example 2:

Menu Source -> Text,
Insert the "Différence" string in the 'Text' area
Click on Apply

In the result appears "Output Message" window with the following log:

Generic Warning: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Common/Core/vtkUnicodeString.cxx, line 183
vtkUnicodeString::from_utf8(): not a valid UTF-8 string.


Generic Warning: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Common/Core/vtkUnicodeString.cxx, line 183
vtkUnicodeString::from_utf8(): not a valid UTF-8 string.


Generic Warning: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Common/Core/vtkUnicodeString.cxx, line 183
vtkUnicodeString::from_utf8(): not a valid UTF-8 string.


Generic Warning: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Common/Core/vtkUnicodeString.cxx, line 183
vtkUnicodeString::from_utf8(): not a valid UTF-8 string.


ERROR: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Rendering/FreeType/vtkTextActor.cxx, line 803
vtkTextActor (0x36e3d80): Cannot compute bounding box.


Generic Warning: In /misc/dn20/salome/rnv/PRODUCTS_BUILD/ParaView-4.1.0_SRC/VTK/Common/Core/vtkUnicodeString.cxx, line 183
vtkUnicodeString::from_utf8(): not a valid UTF-8 string.
TagsNo tags attached.
ProjectParaViS
Topic NameFix_utf8_issue_for_StringVectorPropertyWidget
Typeincorrect functionality
Attached Filespng file icon paraview.png [^] (75,428 bytes) 2014-06-18 04:42


png file icon salome.png [^] (125,209 bytes) 2014-06-18 04:43

 Relationships

  Notes
(0032748)
Joachim Pouderoux (developer)
2014-06-05 10:26

Hello,

You can use UTF8 strings using the Python wrapping but you have to use UTF8 hex codes for the accentued chars instead. Example:

  Text1 = Text()
  Text1.Text = 'Diff\xC3\xA9rence'
  RenderView1 = GetRenderView()
  DataRepresentation1 = Show()
  Render()

See for instance this page http://www.utf8-chartable.de [^] The third column shows the hex code you should use as escape sequence.

However, because of the Qt layer, this does not work in the GUI (but you can copy/paste the string with special chars that will be generated from the Python source).
(0032757)
Joachim Pouderoux (developer)
2014-06-06 04:32

In the UI, you can direcly use the UTF8 code using something like ALT+[code in decimal].
Exemple: Diff[ALT 0195][ALT 0169]rence
(0032891)
Vadim SANDLER (reporter)
2014-06-18 04:44

Dear Joachim,

Thank you for explanations.

Indeed, it seems that passing 'Diff\xe9rence' to Python might be not correct, if we want Unicode strings to be shown.
It seems to be more correct to pass "Différence" or "Diff\xC3\xA9rence", and naturally "Différence" is more preferable way.

However, there are still problems with processing Unicode strings in ParaView. I think, when you say "because of the Qt layer, this does not work in the GUI" - it is not quite correct.

Indeed, as all application components like Qt, Python and VTK declare support of Unicode strings, there potentially should not be problems with interchanging data between these components, provided that all is implemented in a coherent way.

For example, please take a look at the snapshot from SALOME application (salome.png). In SALOME we pass data between different application components, including:
- From Qt widget (GUI side) to server process via CORBA bus : here we use QString::toUtf8() to pass data as CORBA::String (i.e. char*) [case 1 on the image]
- From server process back to Qt widget (GUI side) via CORBA bus : here we use QString::fromUtf8() to ensure data is properly converted from char* and shown in GUI [cases 2a and 2b on the image]
- From Qt widget to embedded Python interpreter : via QString::toUtf8() [case 3 on the image]
- From embedded Python interpreter back to GUI (Python's feedback) : via QString::fromUtf8() [case 4 on the image]
- From Qt widget to VTK presentation : QString::toUtf8() [case 5 on the image]

As demonstrated by this example, in all cases unicode string data is shown properly. It is OK when we use either "Différence" or "Diff\xC3\xA9rence" in a Python script.

However, in ParaView it's no OK - with both "Différence" or "Diff\xC3\xA9rence") used in a Python script we see good text in VTK viewer but bad one in Qt widget (see paraview.png). It seems that there is missing QString::fromUtf8() somewhere.
An attempt to pass 'Différence' via "Properties" pane leads to a "vtkUnicodeString::from_utf8(): not a valid UTF-8 string" error. Missing QString::toUtf8()?

Also, it's necessary to mention that the processing of a text in the viewer seems to depend on the way we do this.
For instance, in SALOME, if we use vtkTextActor for displaying text label in a viewer, unicode text is shown correctly (as demonstrated by salome.png).
However, if we use vtkActor2D with vtkTextMapper for the same purpose, the result is bad.
Here is the corresponding code:

  // This is OK...
  myTextActor=vtkTextActor::New();
  myTextActor->SetInput(qtstring.toUtf8().constData());

  // This is KO...
  myTextMapper=vtkTextMapper::New();
  myTextActor=vtkActor2D::New();
  myTextActor->SetMapper(myTextMapper);
  myTextMapper->SetInput(qtstring.toUtf8().constData());

Regards,
Vadim.
(0032907)
Joachim Pouderoux (developer)
2014-06-20 06:06
edited on: 2014-06-20 07:52

Thanks for your detailed answer. I have just pushed a fix for this issue but so far it just fixes the Qt=>Proxy problem. I still need to fix Proxy=>Qt.


http://review.source.kitware.com/#/c/15866/ [^]

Change Ia527552c: Fix pqStringVectorPropertyWidget to support utf8 strings.

The problem is described in ticket 0014729.
UTF8 chars were not correctly converted (toLatin1()) was used
so TextSource for instance were not able to support accentuated
chars.

Change-Id: Ia527552c230351364541ddff2e9a527784bed976

(0032916)
Utkarsh Ayachit (administrator)
2014-06-25 11:22

commit 99501ea43b10463f9fcfe251bc9e05fb35acb5c7
Author: Joachim Pouderoux <joachim.pouderoux@kitware.com>
Date: Fri Jun 20 12:01:32 2014 +0200

    Fix pqStringVectorPropertyWidget to support utf8 strings.
    
    The problem is described in ticket 0014729.
    UTF8 chars were not correctly converted (toLatin1()) was used
    so TextSource for instance were not able to support accentuated
    chars.
    
    Change-Id: Ia527552c230351364541ddff2e9a527784bed976
(0032917)
Utkarsh Ayachit (administrator)
2014-06-25 11:22

This above fix should now address the Proxy==>Qt issue as well.
(0032938)
Utkarsh Ayachit (administrator)
2014-06-27 08:41

SUMMARY
---------------------------------------------
Topics merged into master:
        14790_save_restore_settings_buttons
        14814_fix_cell_point_label_format
        14824_fix_reader_factory
        fix-pv-transposetable-filter
        Fix_utf8_issue_for_StringVectorPropertyWidget
        stage-push-data


---------------------------------------------
Topics reverted from next:
  14784_PythonCatalyst_better_IO_2
(0032989)
Alan Scott (manager)
2014-07-08 15:33

I believe this is fixed. I tested using sources/ text, and the example of Différence. Well written tests.

Tested Linux, master, local server.

 Issue History
Date Modified Username Field Change
2014-05-14 10:42 Vadim SANDLER New Issue
2014-06-05 09:38 Joachim Pouderoux Assigned To => Joachim Pouderoux
2014-06-05 10:26 Joachim Pouderoux Note Added: 0032748
2014-06-05 10:27 Joachim Pouderoux Resolution open => fixed
2014-06-06 04:32 Joachim Pouderoux Note Added: 0032757
2014-06-18 04:42 Vadim SANDLER File Added: paraview.png
2014-06-18 04:43 Vadim SANDLER File Added: salome.png
2014-06-18 04:44 Vadim SANDLER Note Added: 0032891
2014-06-20 06:06 Joachim Pouderoux Note Added: 0032907
2014-06-20 06:06 Joachim Pouderoux Topic Name => Fix_utf8_issue_for_StringVectorPropertyWidget
2014-06-20 06:06 Joachim Pouderoux Reproducibility have not tried => always
2014-06-20 06:06 Joachim Pouderoux Status backlog => active development
2014-06-20 06:24 Joachim Pouderoux Note Edited: 0032907
2014-06-20 07:52 Joachim Pouderoux Note Edited: 0032907
2014-06-25 11:22 Utkarsh Ayachit Note Added: 0032916
2014-06-25 11:22 Utkarsh Ayachit Status active development => gatekeeper review
2014-06-25 11:22 Utkarsh Ayachit Fixed in Version => git-next
2014-06-25 11:22 Utkarsh Ayachit Note Added: 0032917
2014-06-27 08:41 Utkarsh Ayachit Fixed in Version git-next => git-master
2014-06-27 08:41 Utkarsh Ayachit Note Added: 0032938
2014-06-27 09:50 Utkarsh Ayachit Status gatekeeper review => customer review
2014-07-08 15:33 Alan Scott Note Added: 0032989
2014-07-08 15:33 Alan Scott Status customer review => closed
2014-08-29 10:59 Utkarsh Ayachit Fixed in Version git-master => 4.2
2015-01-11 08:17 Utkarsh Ayachit Source_changeset_attached => ParaView master 3ccbba88
2015-01-11 08:18 Utkarsh Ayachit Source_changeset_attached => ParaView master 4a1d88c5


Copyright © 2000 - 2018 MantisBT Team