CID-keyed font support?

[Jiang Jiang]

Hi,

I wonder if this has been discussed before but I have been wondering
if CID-keyed fonts are supported by XeTeX/xdvipdfmx at all.

Recent release of Source Han Sans [1] brought up the question again.

If you install Source Han Sans CN Regular for instance, the following document:

\font\body="Source Han Sans CN" at 10pt \body
测试
\end

won't work:

(./test.tex [1]
** WARNING ** Failed to load ToUnicode CMap for font "SourceHanSansCN-Regular"
** WARNING ** Glyph for CID 16719 missing in font
"/Users/jjgod/Library/Fonts/SourceHanSansCN-Regular.otf".
)

The resulting PDF end up having the wrong glyphs.

I don't mind taking a look but any hints will be appreciated.

- Jiang

[1] http://sourceforge.net/adobe/source-han-sans/wiki/Home/




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:

[Dohyun Kim]

Post by Jiang Jiang
Hi,
I wonder if this has been discussed before but I have been wondering
if CID-keyed fonts are supported by XeTeX/xdvipdfmx at all.
Recent release of Source Han Sans [1] brought up the question again.
\font\body="Source Han Sans CN" at 10pt \body
测试
\end
(./test.tex [1]
** WARNING ** Failed to load ToUnicode CMap for font "SourceHanSansCN-Regular"
** WARNING ** Glyph for CID 16719 missing in font
"/Users/jjgod/Library/Fonts/SourceHanSansCN-Regular.otf".
)
The resulting PDF end up having the wrong glyphs.

With SourceHanSansKR, I experienced the same thing. Let me share my guess.

The number 16719 in the log above must be a GID. But xdvipdfmx
considers it as CID as shown, so there is a discrepancy between xetex
and xdvipdfmx. CID and GID may differ when a cid-keyed font includes
a subset of the glyphs specified by ROS (Adobe-Identity-0 in our
case). See http://blogs.adobe.com/CCJKType/2012/04/cid-vs-gid.html .

So I instead tried the so-called "Pan-CJK" font,
SourceHanSansOTF-1.000.zip downloadable at
http://sourceforge.net/projects/source-han-sans.adobe/files/ . As
this font has full 65535 glyphs under Adobe-Identity-0, CIDs are
identical to GIDs. In this case, however, xdvipdfmx failed to make
PDF with error message as follows:

xdvipdfmx:warning: Failed to load ToUnicode CMap for font
"SourceHanSans-Regular"
(CID:/Users/nomosnomos/Library/Fonts/SourceHanSans-Regular.otf[LJIICX+SourceHanSans-Regular][CIDFontType0]
xdvipdfmx:fatal: sfnt: Freetype failure...
Output file removed.
)
Error 256 (driver return code) generating output;

So it seems that we have to do something with xdvipdfmx and/or its
freetype library. The SourceHanSans fonts are sure to have no problem
as they work quite well on my mac OS.

Best,
Dohyun Kim




--------------------------------------------------
Subscriptions, Archive, and List informat

[Jiang Jiang]

Post by Dohyun Kim

Post by Jiang Jiang
Hi,
I wonder if this has been discussed before but I have been wondering
if CID-keyed fonts are supported by XeTeX/xdvipdfmx at all.
Recent release of Source Han Sans [1] brought up the question again.
\font\body="Source Han Sans CN" at 10pt \body
测试
\end
(./test.tex [1]
** WARNING ** Failed to load ToUnicode CMap for font "SourceHanSansCN-Regular"
** WARNING ** Glyph for CID 16719 missing in font
"/Users/jjgod/Library/Fonts/SourceHanSansCN-Regular.otf".
)
The resulting PDF end up having the wrong glyphs.

With SourceHanSansKR, I experienced the same thing. Let me share my guess.
The number 16719 in the log above must be a GID. But xdvipdfmx
considers it as CID as shown, so there is a discrepancy between xetex
and xdvipdfmx. CID and GID may differ when a cid-keyed font includes
a subset of the glyphs specified by ROS (Adobe-Identity-0 in our
case). See http://blogs.adobe.com/CCJKType/2012/04/cid-vs-gid.html .
So I instead tried the so-called "Pan-CJK" font,
SourceHanSansOTF-1.000.zip downloadable at
http://sourceforge.net/projects/source-han-sans.adobe/files/ . As
this font has full 65535 glyphs under Adobe-Identity-0, CIDs are
identical to GIDs. In this case, however, xdvipdfmx failed to make
xdvipdfmx:warning: Failed to load ToUnicode CMap for font
"SourceHanSans-Regular"
(CID:/Users/nomosnomos/Library/Fonts/SourceHanSans-Regular.otf[LJIICX+SourceHanSans-Regular][CIDFontType0]
xdvipdfmx:fatal: sfnt: Freetype failure...
Output file removed.
)
Error 256 (driver return code) generating output;
So it seems that we have to do something with xdvipdfmx and/or its
freetype library. The SourceHanSans fonts are sure to have no problem
as they work quite well on my mac OS.

In case you or anyone else are interested, I just posted two patches
to xdvipdfmx to fix this problem:
https://gist.github.com/jjgod/c1194a9b371848aaa746

The first patch is to fix text displayed in PDF, the second one is for
correcting the ToUnicode map to make sure text copied out of PDF are
correct.

CCing Peter Breitenlohner, who is the current maintainer of dvipdfm-x.

- Jiang




--------------------------------------------------
Subscriptions, Archive, and List info

[Akira Kakuto]

Dear Jiang Jiang,

Post by Jiang Jiang
In case you or anyone else are interested, I just posted two patches
https://gist.github.com/jjgod/c1194a9b371848aaa746

Thanks very much.
I've succeeded (with 2 or 3 changes for late declarations
of variables, since my compiler is foolish) with NotoSansJP-Regular.
However, with SourceHanSans-Regular (Pan-CJK), I have

xdvipdfmx:fatal: sfnt: Freetype failure...

Is that a bug in FreeType?

Best regards,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi Akira,

Post by Akira Kakuto
Dear Jiang Jiang,

Post by Jiang Jiang
In case you or anyone else are interested, I just posted two patches
https://gist.github.com/jjgod/c1194a9b371848aaa746

Thanks very much.
I've succeeded (with 2 or 3 changes for late declarations
of variables, since my compiler is foolish) with NotoSansJP-Regular.
However, with SourceHanSans-Regular (Pan-CJK), I have
xdvipdfmx:fatal: sfnt: Freetype failure...
Is that a bug in FreeType?

Could be, I haven't tried the Pan-CJK font yet, will take a look.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,

Post by Akira Kakuto

Post by Jiang Jiang
In case you or anyone else are interested, I just posted two patches
https://gist.github.com/jjgod/c1194a9b371848aaa746

I've succeeded (with 2 or 3 changes for late declarations
of variables, since my compiler is foolish) with NotoSansJP-Regular.

However, in many cases of usual fonts, which are ok
in the present xdvipdfmx, the patched xdvipdfmx seems to crash.
For example,

\documentclass{article}
\usepackage{fontspec}
\begin{document}
\fontspec{[msmincho.ttc]}
This is a test. Some Japanese characters.
\end{document}

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,

Sorry, it seems that I was wrong.
The patched xdvipdfmx doesn't crash for the previous example.
But there are many warnings, probably for non-CJK fonts:

xtst -> xtst.pdf
[1][2][3][4][5][6][7][8][9]
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "BernhardModernStd-Roman"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "BernhardModernStd-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "BernhardModernStd-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ACaslonPro-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ACaslonPro-Semibold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ACaslonPro-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "AGaramondPro-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "AGaramondPro-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "AGaramondPro-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "CenturyOldStyleStd-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "CenturyOldStyleStd-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "CenturyOldStyleStd-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ChaparralPro-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ChaparralPro-Semibold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "ChaparralPro-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MinionPro-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MinionPro-Semibold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MinionPro-It"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MyriadPro-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MyriadPro-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "MyriadPro-It"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NewsGothicStd"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NewsGothicStd-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NewsGothicStd-Oblique"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NuevaStd-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NuevaStd-Bold"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "NuevaStd-Italic"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "VivaStd-Regular"
xdvipdfmx:warning: Unable to read OpenType/TrueType Unicode cmap table.
xdvipdfmx:warning: Failed to load ToUnicode CMap for font "VivaStd-Bold"

442061 bytes written

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,
The warnings disappear if I apply an attached
patch: type0.c.diff.

Best,
Akira

[Jiang Jiang]

Post by Akira Kakuto
Dear Jiang Jiang,
The warnings disappear if I apply an attached
patch: type0.c.diff.

I'm not sure if the last section is correct as it will very likely
break the ToUnicode mappings. The idea here is to use the real glyph
ids for "used chars" testing in otf_create_ToUnicode_stream() while
keeping the cids for ToUnicode mapping generation. By using
Type0Font_get_usedchars(font) instead of
Type0Font_get_usedglyphs(font) we will attempt to use the cids for
used chars testing and resulting wrong mappings.

I will take a look at the warnings for non-CJK fonts.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang ,

Post by Jiang Jiang
I will take a look at the warnings for non-CJK fonts.

Thanks.
In addition, I think it is necessary to refine for dvipdfmx.
The patched sources do not compile as "dvipdfmx".
My friend says that the present dvipdfmx can handle
Source Han Sans JP by translating the CMap
UniSourceHanSansJP-UTF32-H into UniSourceHanSansJP-UTF16-H.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi,

Post by Akira Kakuto
Dear Jiang Jiang ,

Post by Jiang Jiang
I will take a look at the warnings for non-CJK fonts.

Thanks.
In addition, I think it is necessary to refine for dvipdfmx.
The patched sources do not compile as "dvipdfmx".

Right, I can look into that.

Post by Akira Kakuto
My friend says that the present dvipdfmx can handle
Source Han Sans JP by translating the CMap
UniSourceHanSansJP-UTF32-H into UniSourceHanSansJP-UTF16-H.

Yes, if you look at the logic there, it was trying to resolve the
ToUnicode CMap by first attempting to load SourceHanSansJP-UTF16-H in
texmf tree (with kpathsea), if this couldn't be found it will try
SourceHanSansJP-UCS2-H instead, if that couldn't be found it will have
to bail out. What I changed there is adding yet another fallback to
use reverse SFNT CMap lookup (otf_create_ToUnicode_stream) to find the
matching Unicode code points for used glyphs.

It will certainly be better if SourceHanSansJP-UTF16-H can be found,
or UniSourceHanSansJP-UTF32-H can be loaded by dvipdfm-x directly (it
should support that anyway). But I just think this font should work
out of the box, without requiring external mapping files to be
present. I suppose we can't find such SourceHanSansJP-UTF16-H file
embedded in the font itself? Then fallback to the brute force method
(otf_create_ToUnicode_stream) is the only way I can think of. Of
course I'm not familiar with CID-keyed fonts at all so I can be wrong.

I'm always curious what would be the better way to gather such
information, do you think I should just ask Dr. Ken Lunde about it?

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,

Post by Jiang Jiang
Yes, if you look at the logic there, it was trying to resolve the
ToUnicode CMap by first attempting to load SourceHanSansJP-UTF16-H in
texmf tree (with kpathsea), if this couldn't be found it will try
SourceHanSansJP-UCS2-H instead, if that couldn't be found it will have
to bail out.

In the case of dvipdfmx, we usually write in a map file,
cid-x.map for example, like
gbm UniSourceHanSansJP-UTF16-H SourceHanSansJP-Regular.otf
where gbm means a tfm: gbm.tfm.

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Akira Kakuto
Dear Jiang Jiang,

Post by Jiang Jiang
Yes, if you look at the logic there, it was trying to resolve the
ToUnicode CMap by first attempting to load SourceHanSansJP-UTF16-H in
texmf tree (with kpathsea), if this couldn't be found it will try
SourceHanSansJP-UCS2-H instead, if that couldn't be found it will have
to bail out.

In the case of dvipdfmx, we usually write in a map file,
cid-x.map for example, like
gbm UniSourceHanSansJP-UTF16-H SourceHanSansJP-Regular.otf
where gbm means a tfm: gbm.tfm.

Yes, I understand that and had spent quite amount of time getting such
map file right, but I consider that approach less "XeTeX-like" as most
XeTeX documents / setup should just work out of the box without
tweaking in each TeX installation. The motivation for my initial patch
is to make such map files unnecessary.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,

Post by Jiang Jiang

Post by Akira Kakuto
In the case of dvipdfmx, we usually write in a map file,
cid-x.map for example, like
gbm UniSourceHanSansJP-UTF16-H SourceHanSansJP-Regular.otf
where gbm means a tfm: gbm.tfm.

Yes, I understand that and had spent quite amount of time getting such
map file right, but I consider that approach less "XeTeX-like" as most
XeTeX documents / setup should just work out of the box without
tweaking in each TeX installation. The motivation for my initial patch
is to make such map files unnecessary.

Yes, that is less XeTeX-like.
In the case of XeTeX-xdvipdfmx, we cannot use the kind of map file
for OpenType fonts. That is, we cannot load OpenType fonts through tfm fonts.

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi Akira,

Post by Akira Kakuto
Dear Jiang Jiang,

Post by Jiang Jiang

Post by Akira Kakuto
In the case of dvipdfmx, we usually write in a map file,
cid-x.map for example, like
gbm UniSourceHanSansJP-UTF16-H SourceHanSansJP-Regular.otf
where gbm means a tfm: gbm.tfm.

Yes, I understand that and had spent quite amount of time getting such
map file right, but I consider that approach less "XeTeX-like" as most
XeTeX documents / setup should just work out of the box without
tweaking in each TeX installation. The motivation for my initial patch
is to make such map files unnecessary.

Yes, that is less XeTeX-like.
In the case of XeTeX-xdvipdfmx, we cannot use the kind of map file
for OpenType fonts. That is, we cannot load OpenType fonts through tfm fonts.

I updated the gist
(https://gist.github.com/jjgod/c1194a9b371848aaa746) again to fix the
following issues:

1. Non-CJK/CID font support.
2. dvipdfmx support.
3. Uploaded a third patch as an attempt to fix the Pan-CJK version of
Source Han Sans, I'm not entirely sure if this is the solution, but it
seems to help me get PDF generated correctly with that font. It's done
by avoiding to read the 65536 entry in CFF char strings, doing that
will trigger the "sent: Freetype failure" error you have seen.

Please give it a try and let me know if there are still issues.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang,

Post by Jiang Jiang
I updated the gist
(https://gist.github.com/jjgod/c1194a9b371848aaa746) again to fix the

Thank you very much.
I think all is ok now. It compiles as the driver dvipdfmx
for engines which output dvi.
It can handle SourceHanSansJP and SourceHanSans (Pan-CJK).
No warnings for non-CJK fonts.

I'm happy if you adopt an attached non-c99.diff to support
old compilers.

Thanks,
Akira

[Jiang Jiang]

Post by Akira Kakuto
Dear Jiang Jiang,

Post by Jiang Jiang
I updated the gist
(https://gist.github.com/jjgod/c1194a9b371848aaa746) again to fix the

Thank you very much.
I think all is ok now. It compiles as the driver dvipdfmx
for engines which output dvi.
It can handle SourceHanSansJP and SourceHanSans (Pan-CJK).
No warnings for non-CJK fonts.
I'm happy if you adopt an attached non-c99.diff to support
old compilers.

Thanks for checking, I applied the patch with some slight
modifications to make it simpler and updated the gist.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi Khaled,

Since Peter is not responding, what do you think of this patch? Should
I go ahead and commit this to TeXLive repository? People had been
using it for a few days with various CID-keyed fonts and haven't
reported any more issue.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Jiang Jiang,

Post by Jiang Jiang
Since Peter is not responding, what do you think of this patch? Should
I go ahead and commit this to TeXLive repository? People had been
using it for a few days with various CID-keyed fonts and haven't
reported any more issue.

I'll apply your patch. Peter will correct if it is necessary.

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Peter Breitenlohner]

Since Peter is not responding, ....

Hi Jiang Jiang,

that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

Regards
Peter



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang
Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger) and
this of course is conflicting with your changes. I had to revert them
locally for now, and will need to figure out how to re-do it again since
the fc_face member of the various font structure is almost gone.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Khaled Hosny

Post by Jiang Jiang
Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger) and
this of course is conflicting with your changes. I had to revert them
locally for now, and will need to figure out how to re-do it again since
the fc_face member of the various font structure is almost gone.

There has to be a FT_Face somewhere, right?

Let me know if you need help with that.

There are issues with CID-keyed fonts ToUnicode handling in dvipdfmx
with external CMap provided as well, I don't know if it's worth to fix
it though.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang

Post by Khaled Hosny

Post by Jiang Jiang
Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger) and
this of course is conflicting with your changes. I had to revert them
locally for now, and will need to figure out how to re-do it again since
the fc_face member of the various font structure is almost gone.

There has to be a FT_Face somewhere, right?

Probably not, at least unless absolutely required.

Post by Jiang Jiang
Let me know if you need help with that.

Sure.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Khaled Hosny

Post by Jiang Jiang

Post by Khaled Hosny

Post by Jiang Jiang
Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger) and
this of course is conflicting with your changes. I had to revert them
locally for now, and will need to figure out how to re-do it again since
the fc_face member of the various font structure is almost gone.

There has to be a FT_Face somewhere, right?

Probably not, at least unless absolutely required.

I'm not sure it's a good idea. I looked at your changes and I'd rather
it to be the other way around, remove the legacy dvipdfm-x code and
replace it with FreeType calls instead. At a glance I felt that
FreeType code is way better maintained and in a much better shape than
dvipdfm-x code, I can't remember how many times I see "FIXME" and
"This should be fixed" in dvipdfm-x code base. Take CFF and CID font
support for instance, I suspect that we will have to spent 10 times
more effort to correct the current dvipdfm-x code while FreeType has
high quality code (some contributed by Adobe themselves) ready for
use. While unifying the dvipdfmx and xdvipdfmx is a goal worth
pursuing I doubt it will actually bring a higher quality version to
XeTeX users in the next release.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Since the old patches no longer apply, I updated
https://gist.github.com/jjgod/c1194a9b371848aaa746 with first patch
rewritten.

As you all can see, it is possible to do it in legacy dvipdfm-x code,
just more tedious, I'm not sure what benefit that gives. Then again, I
don't care that much as long as it works.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang

Post by Khaled Hosny

Post by Jiang Jiang

Post by Khaled Hosny

Post by Jiang Jiang
Hi,

Post by Akira Kakuto

Since Peter is not responding, ....

Hi Jiang Jiang,
that was on purpose because I am not the maintainer and don't know too much
abaout that program. All I did were purely technical things such as
identifying identical code in the two programs, code cleanup (C language
bugs and compiler warnings) and general TeX Live infrastructure.

OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger) and
this of course is conflicting with your changes. I had to revert them
locally for now, and will need to figure out how to re-do it again since
the fc_face member of the various font structure is almost gone.

There has to be a FT_Face somewhere, right?

Probably not, at least unless absolutely required.

I'm not sure it's a good idea. I looked at your changes and I'd rather
it to be the other way around, remove the legacy dvipdfm-x code and
replace it with FreeType calls instead. At a glance I felt that
FreeType code is way better maintained and in a much better shape than
dvipdfm-x code, I can't remember how many times I see "FIXME" and
"This should be fixed" in dvipdfm-x code base.

I'd like to move into that direction as well, but I need to make sure
the unified driver works as intended first and it seemed that unifying
on the side of dvipdfmx that is more feature complete than xdvipdfmx is
easier on the short run. Once the unified driver is working fine, we can
start replacing parts of the dvipdfmx code with equivalent FreeType
API's (I'd like to note that not all parts of FreeType that well
maintained; I have found several bugs while working on XeTeX).

Post by Jiang Jiang
Take CFF and CID font
support for instance, I suspect that we will have to spent 10 times
more effort to correct the current dvipdfm-x code while FreeType has
high quality code (some contributed by Adobe themselves) ready for
use. While unifying the dvipdfmx and xdvipdfmx is a goal worth
pursuing I doubt it will actually bring a higher quality version to
XeTeX users in the next release.

Let hope we have enough time before the next release to make sure it
is :)

Regards,
Khaled




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xete

[Akira Kakuto]

Hi Khaled,

Let hope we have enough time before the next release to make sure it is :)

xdvipdfmx r34745 (after applying the patch by Jiang) sometimes
crashes at

--- cff.c.orig Mon Jul 28 08:50:32 2014
+++ cff.c Mon Jul 28 09:57:29 2014
@@ -1172,12 +1172,12 @@
break;
case 2:
if (charset->data.range2)
- RELEASE(charset->data.range2);
+ RELEASE(charset->data.range2); /* crash sometimes */
break;
default:
break;
}
- RELEASE(charset);
+ RELEASE(charset); /* crash sometimes */
}
}


An example (sample.tex) is in an attached data.tar.gz.

xdvipdfmx r34722 (before applying the patch by Jiang) seems to
work, but with many strange warnings (mess2.txt in data.tar.gz),
compared with r34711 which consists of the original Jonathan's
code (mess1.txt in data.tar.gz).

I think we should go back to r34711.
I find no reason to unify dvipdfmx and xdvipdfmx further.
The issue noted by Ulrike should be solved at package level.

Best,
Akira

[Khaled Hosny]

Post by Jiang Jiang
Hi Khaled,

Let hope we have enough time before the next release to make sure it is :)

xdvipdfmx r34745 (after applying the patch by Jiang) sometimes
crashes at
--- cff.c.orig Mon Jul 28 08:50:32 2014
+++ cff.c Mon Jul 28 09:57:29 2014
@@ -1172,12 +1172,12 @@
break;
if (charset->data.range2)
- RELEASE(charset->data.range2);
+ RELEASE(charset->data.range2); /* crash sometimes */
break;
break;
}
- RELEASE(charset);
+ RELEASE(charset); /* crash sometimes */
}
}
An example (sample.tex) is in an attached data.tar.gz.
xdvipdfmx r34722 (before applying the patch by Jiang) seems to
work, but with many strange warnings (mess2.txt in data.tar.gz),
compared with r34711 which consists of the original Jonathan's
code (mess1.txt in data.tar.gz).
I think we should go back to r34711.
I find no reason to unify dvipdfmx and xdvipdfmx further.
The issue noted by Ulrike should be solved at package level.

This is a development tree after all, so bugs are to be expected. Lets
debug those issues first then decide what we want to do.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled,

Post by Khaled Hosny
This is a development tree after all, so bugs are to be expected. Lets
debug those issues first then decide what we want to do.

OK, agreed. Thank you very much.

Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang
Hi Khaled,

Post by Khaled Hosny
This is a development tree after all, so bugs are to be expected. Lets
debug those issues first then decide what we want to do.

OK, agreed. Thank you very much.

The /rsrc warning seems harmless, but I'm still investigating what is
causing it. The CMap issue should be fixed now.

(It is the holidays here in Egypt, so I might be a bit slow to respond).

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Khaled Hosny

Post by Jiang Jiang
Hi Khaled,

Post by Khaled Hosny
This is a development tree after all, so bugs are to be expected. Lets
debug those issues first then decide what we want to do.

OK, agreed. Thank you very much.

The /rsrc warning seems harmless, but I'm still investigating what is
causing it. The CMap issue should be fixed now.

It's because when xdvipdfmx is trying to load an non-CID OpenType
font, CIDFont_cache_find() will first attempt to load such font as a
Type2 font, which will trigger such a OS X resource fork access,
previously we had sfnt_open to replace that for XETEX:

-#ifdef XETEX
- sfont = sfnt_open(font->ft_face, SFNT_TYPE_TTC | SFNT_TYPE_TRUETYPE);
-#else
fp = DPXFOPEN(font->ident, DPX_RES_TYPE_TTFONT);
if (!fp) {
fp = DPXFOPEN(font->ident, DPX_RES_TYPE_DFONT);
@@ -553,7 +550,6 @@ CIDFont_type2_dofont (CIDFont *font)
} else {
sfont = sfnt_open(fp);
}
-#endif

The backtrace is as follow:

* frame #0: 0x000000010009c87b
xdvipdfmx`qcheck_filetype(fqpn=0x000000010072f440,
type=DPX_RES_TYPE_DFONT) + 59 at dpxfile.c:1170
frame #1: 0x000000010009c827
xdvipdfmx`dpx_find_dfont_file(filename=<unavailable>) + 151 at
dpxfile.c:764
frame #2: 0x000000010009c065
xdvipdfmx`dpx_open_file(filename=<unavailable>, type=<unavailable>) +
229 at dpxfile.c:446
frame #3: 0x0000000100098d6d
xdvipdfmx`CIDFont_type2_open(font=0x000000010072e980,
name=0x000000010072eb10, cmap_csi=0x0000000000000000,
opt=0x000000010072e950) + 93 at cidtype2.c:943
frame #4: 0x00000001000928a3
xdvipdfmx`CIDFont_cache_find(map_name=0x000000010072eb10,
cmap_csi=0x0000000000000000, fmap_opt=0x000000010072ea48) + 1667 at
cid.c:633
frame #5: 0x00000001000d37fc
xdvipdfmx`Type0Font_cache_find(map_name=0x000000010072eb10,
cmap_id=<unavailable>, fmap_opt=0x000000010072ea48) + 108 at
type0.c:421
frame #6: 0x00000001000ab1af
xdvipdfmx`pdf_font_findresource(tex_name=0x000000010072ced0,
font_scale=<unavailable>, mrec=0x000000010072ea20) + 479 at
pdffont.c:659
frame #7: 0x00000001000169b0
xdvipdfmx`pdf_dev_locate_font(font_name=0x000000010072ced0,
ptsize=1132462) + 592 at pdfdev.c:1504
frame #8: 0x0000000100009001 xdvipdfmx`do_fnt [inlined]
dvi_locate_native_font(ps_name=<unavailable>, ptsize=1132462,
layout_dir=<unavailable>, extend=<unavailable>, slant=<unavailable>,
embolden=<unavailable>) + 314 at dvi.c:1062
frame #9: 0x0000000100008ec7
xdvipdfmx`do_fnt(tex_id=<unavailable>) + 167 at dvi.c:1784
frame #10: 0x00000001000065ec
xdvipdfmx`dvi_do_page(n=<unavailable>, paper_width=<unavailable>,
paper_height=<unavailable>, hmargin=<unavailable>,
vmargin=<unavailable>) + 140 at dvi.c:2243
frame #11: 0x000000010000d1c2 xdvipdfmx`main [inlined]
do_dvi_pages + 881 at dvipdfmx.c:816
frame #12: 0x000000010000ce51 xdvipdfmx`main(argc=<unavailable>,
argv=<unavailable>) + 2065 at dvipdfmx.c:1064

r34758 should fix this.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang

Post by Khaled Hosny

Post by Jiang Jiang
Hi Khaled,

Post by Khaled Hosny
This is a development tree after all, so bugs are to be expected. Lets
debug those issues first then decide what we want to do.

OK, agreed. Thank you very much.

The /rsrc warning seems harmless, but I'm still investigating what is
causing it. The CMap issue should be fixed now.

[...]

Post by Jiang Jiang
r34758 should fix this.

Thanks for taking care of it.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,

I think r34760 works fine. Thanks very much for your work.

Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Jiang Jiang
Hi Khaled,

Let hope we have enough time before the next release to make sure it is :)

xdvipdfmx r34745 (after applying the patch by Jiang) sometimes
crashes at
--- cff.c.orig Mon Jul 28 08:50:32 2014
+++ cff.c Mon Jul 28 09:57:29 2014
@@ -1172,12 +1172,12 @@
break;
if (charset->data.range2)
- RELEASE(charset->data.range2);
+ RELEASE(charset->data.range2); /* crash sometimes */
break;
break;
}
- RELEASE(charset);
+ RELEASE(charset); /* crash sometimes */
}
}
An example (sample.tex) is in an attached data.tar.gz.
xdvipdfmx r34722 (before applying the patch by Jiang) seems to
work, but with many strange warnings (mess2.txt in data.tar.gz),
compared with r34711 which consists of the original Jonathan's
code (mess1.txt in data.tar.gz).

Thank you Akira, I can reproduce this, will take a look.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi Akira,

Post by Jiang Jiang
Thank you Akira, I can reproduce this, will take a look.

https://gist.github.com/jjgod/cf39032b839bb3542107 should fix the
crashes for you.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,

In r34750, there are no crashes, and SourceHanSans... work, thanks Jiang.
Graphics in R-to-L mode is OK, thanks Khaled.

I attach a sample.tex, for which there are many strange warnings,
and I cannot paste some lines of text in an editor, that is, incorrect toUnicode.

If I use r34711, I can copy & paste all lines.

Thanks,
Akira

[Ulrike Fischer]

Post by Khaled Hosny

Post by Jiang Jiang
OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger)

Did you saw the discussion on the Texlive list regarding the
problems with loading otf-fonts through map-file entries with
xdvipdfmx?

("(x)dvipdfmx and fully embeded fonts in pdftex.map")
--
Ulrike Fischer
http://www.troubleshooting-tex.de/




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Ulrike Fischer

Post by Khaled Hosny

Post by Jiang Jiang
OK, sorry about that. So who is the maintainer of dvipdfm-x then?

I partly do, and it happens that I was in the middle of some major
refactoring removing most of the different font loading paths between
dvipdfmx and xdvipdfmx (in preparation of their eventual merger)

Did you saw the discussion on the Texlive list regarding the
problems with loading otf-fonts through map-file entries with
xdvipdfmx?

Yes, that is the issue I'm tackling.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Ulrike Fischer]

Post by Khaled Hosny

Post by Ulrike Fischer
Did you saw the discussion on the Texlive list regarding the
problems with loading otf-fonts through map-file entries with
xdvipdfmx?

Yes, that is the issue I'm tackling.

Good ;-)
--
Ulrike Fischer
http://www.troubleshooting-tex.de/




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Post by Ulrike Fischer

Post by Khaled Hosny

Post by Ulrike Fischer
Did you saw the discussion on the Texlive list regarding the
problems with loading otf-fonts through map-file entries with
xdvipdfmx?

Yes, that is the issue I'm tackling.

Good ;-)

I think Khaled has fixed the issue (r34722).
It seems that he is temporarily disabling the support of
CID-keyed OpenType fonts by Jiang.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Dear Jiang Jiang ,

Post by Jiang Jiang
I will take a look at the warnings for non-CJK fonts.

If I use your
Type0Font_create_ToUnicode_stream(font);
for
case CIDFONT_TYPE2:
and the original
otf_create_ToUnicode_stream()
for
default:
there are no warnings, and I could copy and paste
Japanese character strings.

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Akira Kakuto
Dear Jiang Jiang ,

Post by Jiang Jiang
I will take a look at the warnings for non-CJK fonts.

If I use your
Type0Font_create_ToUnicode_stream(font);
for
and the original
otf_create_ToUnicode_stream()
for
there are no warnings, and I could copy and paste
Japanese character strings.

Interesting, what I got from Source Han Sans CN Regular is the default
case, the font doesn't have any extra flag (eg. CIDFONT_FLAG_TYPE1C or
CIDFONT_FLAG_TYPE1). That's why I changed the code there, I guess I
will have to check the Japanese font and see if it goes to the
CIDFONT_TYPE2 case.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Jiang Jiang,

Post by Akira Kakuto
I'll apply your patch. Peter will correct if it is necessary.

Done (r34700). If something is wrong, please report.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Post by Akira Kakuto
Hi Jiang Jiang,

Post by Akira Kakuto
I'll apply your patch. Peter will correct if it is necessary.

Done (r34700). If something is wrong, please report.

Thanks.

Do you by any chance know how regular or how soon will TeXLive update
to receive patched binary packages?

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Jiang Jiang,

Post by Jiang Jiang
Do you by any chance know how regular or how soon will TeXLive update
to receive patched binary packages?

Binaries are in principle not updated until the next year.
Thus interested users should compile it themselves.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,

A japenese user reported that the new dvipdfmx crashed
in the following example:

%
% sample.tex
% (dvipdfmx r34778 crashes)
% (dvipdfmx r34711 works fine)
%
% platex sample
% dvipdfmx sample
%
\documentclass{tarticle}
\usepackage{otf}
\begin{document}
This is a test. \bou{This is a test.}
\end{document}


I've confirmed that r34778 crashes.
I cannot find the location where the crash occurs.
It is very unstable:
Sometimes it writes a pdf and crashes, while
it crashes without writing a pdf in other cases.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Akira Kakuto
Hi Khaled, Jiang,
A japenese user reported that the new dvipdfmx crashed
%
% sample.tex
% (dvipdfmx r34778 crashes)
% (dvipdfmx r34711 works fine)
%
% platex sample
% dvipdfmx sample
%
\documentclass{tarticle}
\usepackage{otf}
\begin{document}
This is a test. \bou{This is a test.}
\end{document}
I've confirmed that r34778 crashes.
I cannot find the location where the crash occurs.
Sometimes it writes a pdf and crashes, while
it crashes without writing a pdf in other cases.

Can you send me the DVI file? (I just discovered that, for some reason, I
don't have any of the CJK collections installed. I'm installing them
now but my connection is not that fast...).

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Khaled Hosny]

Post by Jiang Jiang
Hi Khaled,

Post by Khaled Hosny

Post by Akira Kakuto
--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 09:06:25 2014
@@ -132,7 +132,7 @@
if (font->descriptor)
ERROR("%s: FontDescriptor unexpected for Type0 font.", TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
- RELEASE(font->used_chars);
+ RELEASE(font->used_chars); /* The crash occurs here */
if (font->used_glyphs)
RELEASE(font->used_glyphs);
if (font->encoding)
In the case of the example on Windows, font->used_chars seems to be
broken, though it is not NULL.

Can you send me the DVI file? (I just discovered that, for some reason, I
don't have any of the CJK collections installed. I'm installing them
now but my connection is not that fast...).

I attach sample.tar.gz.
Japanese font is needed to typeset sample.dvi, though Japanese characters are
not written explicitly.
( If I remove the above RELEASE(font->used_chars); the example works ok on Windows.)

I can not reproduce the crash here, so I'll leave it to Jiang to
comment.

Regards,
Khaled



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled,

I can not reproduce the crash here, so I'll leave it to Jiang to comment.

After some tests, I found the following:

It depends on the type of Japanese font used.

(1) Not embed Japanese font
There is no crash.
The resulting PDF is fine.

(2) Embed TrueType Japanese font
There is no crash.
The resulting PDF is fine.

(3) Embed OpenType Japanese font
The crash occurs.

I suspect your case is (1) or (2).
In the case of (3), font->used_glyphs, a new data
introduced by Jiang, may be broken.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled,

Post by Khaled Hosny

Post by Akira Kakuto
--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 09:06:25 2014
@@ -132,7 +132,7 @@
if (font->descriptor)
ERROR("%s: FontDescriptor unexpected for Type0 font.", TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
- RELEASE(font->used_chars);
+ RELEASE(font->used_chars); /* The crash occurs here */
if (font->used_glyphs)
RELEASE(font->used_glyphs);
if (font->encoding)
In the case of the example on Windows, font->used_chars seems to be
broken, though it is not NULL.

Can you send me the DVI file? (I just discovered that, for some reason, I
don't have any of the CJK collections installed. I'm installing them
now but my connection is not that fast...).

I attach sample.tar.gz.
Japanese font is needed to typeset sample.dvi, though Japanese characters are
not written explicitly.

( If I remove the above RELEASE(font->used_chars); the example works ok on Windows.)

Thanks,
Akira

[Akira Kakuto]

Hi Khaled, Jiang,

Post by Akira Kakuto
I cannot find the location where the crash occurs.

If I remove
pdf_close_fonts ();
in pdf_close_document (void) in pdfdoc.c,
the crash disappears, of course the resulting pdf is bad.
Thus I believe pdf_close_fonts() in pdffont.c must be
the reason of the crash on Windows.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,

Post by Akira Kakuto

Post by Akira Kakuto
I cannot find the location where the crash occurs.

If I remove
pdf_close_fonts ();
in pdf_close_document (void) in pdfdoc.c,
the crash disappears, of course the resulting pdf is bad.
Thus I believe pdf_close_fonts() in pdffont.c must be
the reason of the crash on Windows.

I have found the location where the crash occurs:


--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 09:06:25 2014
@@ -132,7 +132,7 @@
if (font->descriptor)
ERROR("%s: FontDescriptor unexpected for Type0 font.", TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
- RELEASE(font->used_chars);
+ RELEASE(font->used_chars); /* The crash occurs here */
if (font->used_glyphs)
RELEASE(font->used_glyphs);
if (font->encoding)


In the case of the example on Windows, font->used_chars seems to be
broken, though it is not NULL.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,
I was completely wrong. Actually, the crash on Windows occurs
at the following:

--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 10:18:35 2014
@@ -133,8 +133,11 @@
ERROR("%s: FontDescriptor unexpected for Type0 font.", TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
RELEASE(font->used_chars);
+/*
+ crashes on windows for an example
if (font->used_glyphs)
RELEASE(font->used_glyphs);
+*/
if (font->encoding)
RELEASE(font->encoding);
if (font->fontname)


It may be a bug in MSVC compiler if it is not reproduced on
other platforms.

Best,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Akira Kakuto]

Hi Khaled, Jiang,

I can not reproduce the crash here, so I'll leave it to Jiang to comment.

I've found the reason and fixed in r34790.

The changed line is:
--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 11:59:44 2014
@@ -133,7 +133,7 @@
ERROR("%s: FontDescriptor unexpected for Type0 font.", TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
RELEASE(font->used_chars);
- if (font->used_glyphs)
+ if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_glyphs)
RELEASE(font->used_glyphs);
if (font->encoding)
RELEASE(font->encoding);--- type0.c.orig Mon Jul 28 19:38:13 2014

Thanks,
Akira




--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex

[Jiang Jiang]

Hi,

Post by Akira Kakuto
Hi Khaled, Jiang,

I can not reproduce the crash here, so I'll leave it to Jiang to comment.

I've found the reason and fixed in r34790.
--- type0.c.orig Mon Jul 28 19:38:13 2014
+++ type0.c Fri Aug 01 11:59:44 2014
@@ -133,7 +133,7 @@
ERROR("%s: FontDescriptor unexpected for Type0 font.",
TYPE0FONT_DEBUG_STR);
if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_chars)
RELEASE(font->used_chars);
- if (font->used_glyphs)
+ if (!(font->flags & FLAG_USED_CHARS_SHARED) && font->used_glyphs)
RELEASE(font->used_glyphs);
if (font->encoding)
RELEASE(font->encoding);--- type0.c.orig Mon Jul 28 19:38:13 2014

Thanks a lot Akira, this looks like the correct fix. I'm sorry that I
missed the flag in my earlier patch.

- Jiang



--------------------------------------------------
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex