Hi Rikard, Hi all !
[*update on the previous conversation*
Rikard has notified us that Swedish authorities tend to use more and more PDF forms created in a way that PDF readers other than Adobe Acrobat Reader (hereafter AA Reader) can open it but display :
"To view the full contents of this document, you need a later version of the PDF viewer. You can upgrade to the latest version of Adobe Reader from www.adobe.com/products/acrobat/readstep2.html For further support, go to www.adobe.com/support/products/acrreader.html"
an example: http://polisen.se/Vastra_Gotaland/sv/Service/Blanketter/Tillstand/Vanliga-bl...
A developper from Evince helped us determine the source of the problem :
- the PDF is specially crafted so you won't be able to see it unless you are using Adobe reader and a new version... - the institution seems to be using Adobe Live Cycling to create the PDFs.
We think it a good idea to contact the institutions to make them aware that this is a problem... Rikard has declared himself ready to contact them, but we might need some help in preparation of the contact]
Rikard already contacted the Swedish police who was quite little understanding :
On 01/19/2011 01:26 PM, Rikard Fröberg wrote:
I actually contacted the Swedish Police on this matter and described the
situation thoroughly to them, explaining each step and the implications of them offering applications in a form that requires particular software, proprietary no less, and got a confused reply where they explained: "Yes, we are aware. That's why we changed to PDF recently." Is there a wiki or so, on which I could create a sub page for this PDF
forms problem? I could start listing Swedish institutions offering this crippled format and provide example links there.
I.e. they didn't understand that the PDFs they create cause a problem similar to when they had *other* file formats before. Sigh.
So, first, Rikard, we would need to know exactly what you explained to them, so that we know what arguments are not convincing for them.
From what I see so far, *We need to explain them* :
- PDF is an Open Standard [1] - Open Standard are precisely here to make sure several programs from several developpers / companies can open then, i.e. to make sure to ensure *independence from vendor*, *maximal accessibility for the public* and *compatibility*. - In creating / editing PDFs that can only be open by AA Reader deprives the Standard from its advantages, but worse, it forces people into using one program, into buying it from one vendor (being locked with him), and the fact that it is free of charge is no parameter here. - They could improve their accessibility in creating forms that are not crafted, that can be open by any program designed to open the standard. - we can help them to figure out how to do that
*we should not say* : - The way they create their form is wrong -> they would probably react not be very listening if we start telling them they are wrong, we should rather tell them they can be better.
*What we need to search in preparation*:
- Free Software PDF readers are not very performant with forms. They can usually open them, but not save the information entered, and not perform a check-up routine. We need to determine which software are good at *editing* PDFs that can be open by all readers. It would be better here to have Free Software to suggest, in case they aren't, I guess it would already be an improvement to have institutions publish forms that can be open by Free Software even if they are created by a proprietary one... - Rikard suggested there were possibilities to create forms with Open Office or libre Office. Nico here tested Open Office, it worked for what concerns the form functions, but we couldn't figure out how to design the layout we wanted for the PDF... Some research here would be good before recommending these options - other options would be good.
- @Rikard : there is the possibility to create a wiki page on FSFE's wiki (wiki.fsfe.org). If you don't have an account there, either one of the Fellows in Sweden could help you create the list of institutions, or you can create a guest account [2].
*Do you have other ideas, comments on how we can procede here*?
Thanks for your involvement here,
Regards
Maëlle
[1] http://pdfreaders.org/os.en.html [2] https://wiki.fsfe.org/UserGuide#Guest_accounts
Hi.
On the subject of free software alternatives (for generating PDF files) ...
I've been producing PDFs using the Java library iText. It recently got updated with code that made sure the produced PDF file was a PDF/A, which is an ISO standard (subset of) PDF targeted towards the kind of openness we're all aiming for here. (Please correct me if I'm wrong.)
http://itextpdf.com/ (the website's been updated I can see)
Two notes: * I've not tried to create forms with iText.
* When you get into details, and also starting off looking for some kind of overview, I find iText to be not very well documented. (Giving professional training and selling non-free books is how the author of the iText library sponsors his work on this library.) On the other hand, the library has been used in many situations and seems to cover most (if not all) parts of (som version of) the PDF spec.
iText is a _library_, and there's no GUI at all. I use it to produce printing-ready PDFs in a "web to print" solution at webbtryck.se and all printing shops I've been in contact with have been able to just "push the button" and get the document printed properly in their high-end printers/machines. (My source code is hosted on http://source.wendt.se/ to comply with iText's licensing, AGPL).
Sorry if this is old news to you all. :)
/ Fredrik
fre 2011-01-21 klockan 11:35 +0100 skrev Maelle Costa:
Hi Rikard, Hi all !
[*update on the previous conversation*
Rikard has notified us that Swedish authorities tend to use more and more PDF forms created in a way that PDF readers other than Adobe Acrobat Reader (hereafter AA Reader) can open it but display :
"To view the full contents of this document, you need a later version of the PDF viewer. You can upgrade to the latest version of Adobe Reader from www.adobe.com/products/acrobat/readstep2.html For further support, go to www.adobe.com/support/products/acrreader.html"
an example: http://polisen.se/Vastra_Gotaland/sv/Service/Blanketter/Tillstand/Vanliga-bl...
A developper from Evince helped us determine the source of the problem :
- the PDF is specially crafted so you won't be able to see it unless you
are using Adobe reader and a new version...
- the institution seems to be using Adobe Live Cycling to create the
PDFs.
We think it a good idea to contact the institutions to make them aware that this is a problem... Rikard has declared himself ready to contact them, but we might need some help in preparation of the contact]
Rikard already contacted the Swedish police who was quite little understanding :
On 01/19/2011 01:26 PM, Rikard Fröberg wrote:
I actually contacted the Swedish Police on this matter and described the
situation thoroughly to them, explaining each step and the implications of them offering applications in a form that requires particular software, proprietary no less, and got a confused reply where they explained: "Yes, we are aware. That's why we changed to PDF recently." Is there a wiki or so, on which I could create a sub page for this PDF
forms problem? I could start listing Swedish institutions offering this crippled format and provide example links there.
I.e. they didn't understand that the PDFs they create cause a problem similar to when they had *other* file formats before. Sigh.
So, first, Rikard, we would need to know exactly what you explained to them, so that we know what arguments are not convincing for them.
From what I see so far, *We need to explain them* :
- PDF is an Open Standard [1]
- Open Standard are precisely here to make sure several programs from
several developpers / companies can open then, i.e. to make sure to ensure *independence from vendor*, *maximal accessibility for the public* and *compatibility*.
- In creating / editing PDFs that can only be open by AA Reader deprives
the Standard from its advantages, but worse, it forces people into using one program, into buying it from one vendor (being locked with him), and the fact that it is free of charge is no parameter here.
- They could improve their accessibility in creating forms that are not
crafted, that can be open by any program designed to open the standard.
- we can help them to figure out how to do that
*we should not say* :
- The way they create their form is wrong -> they would probably react
not be very listening if we start telling them they are wrong, we should rather tell them they can be better.
*What we need to search in preparation*:
- Free Software PDF readers are not very performant with forms. They can
usually open them, but not save the information entered, and not perform a check-up routine. We need to determine which software are good at *editing* PDFs that can be open by all readers. It would be better here to have Free Software to suggest, in case they aren't, I guess it would already be an improvement to have institutions publish forms that can be open by Free Software even if they are created by a proprietary one...
- Rikard suggested there were possibilities to create forms with Open
Office or libre Office. Nico here tested Open Office, it worked for what concerns the form functions, but we couldn't figure out how to design the layout we wanted for the PDF... Some research here would be good before recommending these options
other options would be good.
@Rikard : there is the possibility to create a wiki page on FSFE's
wiki (wiki.fsfe.org). If you don't have an account there, either one of the Fellows in Sweden could help you create the list of institutions, or you can create a guest account [2].
*Do you have other ideas, comments on how we can procede here*?
Thanks for your involvement here,
Regards
Maëlle
[1] http://pdfreaders.org/os.en.html [2] https://wiki.fsfe.org/UserGuide#Guest_accounts
On 22/01/2011 19:45, Fredrik Wendt wrote:
Hi.
On the subject of free software alternatives (for generating PDF files) ...
I've been producing PDFs using the Java library iText. It recently got updated with code that made sure the produced PDF file was a PDF/A, which is an ISO standard (subset of) PDF targeted towards the kind of openness we're all aiming for here. (Please correct me if I'm wrong.)
http://itextpdf.com/ (the website's been updated I can see)
Two notes:
I've not tried to create forms with iText.
When you get into details, and also starting off looking for some kind
of overview, I find iText to be not very well documented. (Giving professional training and selling non-free books is how the author of the iText library sponsors his work on this library.) On the other hand, the library has been used in many situations and seems to cover most (if not all) parts of (som version of) the PDF spec.
iText is a _library_, and there's no GUI at all. I use it to produce printing-ready PDFs in a "web to print" solution at webbtryck.se and all printing shops I've been in contact with have been able to just "push the button" and get the document printed properly in their high-end printers/machines. (My source code is hosted on http://source.wendt.se/ to comply with iText's licensing, AGPL).
Sorry if this is old news to you all. :)
Absolutely not, at least I did not know anything about this library or your work ;). Thanks a lot for the precise information you provide, and sorry for the far-too-long answer time!
Your project sounds pretty interesting indeed. I can't tell much from http://source.wendt.se/, apart that there seem to be a web GUI? Is your software already usable, and how, and with which features, and by which kind of user?
Thanks, Nico
fre 2011-02-25 klockan 14:25 +0100 skrev Nicolas JEAN:
On 22/01/2011 19:45, Fredrik Wendt wrote:
...
iText is a _library_, [...]
Sorry if this is old news to you all. :)
Absolutely not, at least I did not know anything about this library or your work ;). Thanks a lot for the precise information you provide, and sorry for the far-too-long answer time!
Your project sounds pretty interesting indeed. I can't tell much from http://source.wendt.se/, apart that there seem to be a web GUI? Is your software already usable, and how, and with which features, and by which kind of user?
Well, my work is very limited and targeted towards real estate firms (or any corporation with a fairly fixed set of templates to use). It is being used and works well, though entering new clients, accounts and templates means you need to INSERT all details by hand into a Postgres SQL database (font, color, size, x&y coordinates in pt (points, not pixels) ...). (source.wendt.se contains other projects as well.) The documentation is non-existent as I did all of this over two weekends for a former colleague - with no intention of ever being useful to someone else really. (The code is fairly well unittested though.)
I don't think my work in particular is very interesting. iText on the other hand is - it's a great tool for generating Open PDF files and I think it deserves being promoted as an option to Adobe's server side PDF generation software.
/ Fredrik
On Fri, 2011-01-21 at 11:35 +0100, Maelle Costa wrote:
Hi Rikard, Hi all !
Hi all (added Stian Rødven Eide to the CC-list, as Stian has worked with the PDF readers project and also knows about creating PDF forms using free software)!
[problem description]
http://polisen.se/Vastra_Gotaland/sv/Service/Blanketter/Tillstand/Vanliga-bl...
A developper from Evince helped us determine the source of the problem :
- the PDF is specially crafted so you won't be able to see it unless you
are using Adobe reader and a new version...
- the institution seems to be using Adobe Live Cycling to create the
PDFs.
[my contacting the Swedish police on their PDF forms]
So, first, Rikard, we would need to know exactly what you explained to them, so that we know what arguments are not convincing for them.
It's not so much that they weren't convinced by my arguments, they rather didn't understand them and thought just using PDF was already the answer to my problems. What I told them was that the version of PDF that their documents had, was forcing me to install one particular proprietary reader (AA Reader). I also added my views on why this was bad.
From what I see so far, *We need to explain them* :
- PDF is an Open Standard [1]
- Open Standard are precisely here to make sure several programs from
several developpers / companies can open then, i.e. to make sure to ensure *independence from vendor*, *maximal accessibility for the public* and *compatibility*.
- In creating / editing PDFs that can only be open by AA Reader deprives
the Standard from its advantages, but worse, it forces people into using one program, into buying it from one vendor (being locked with him), and the fact that it is free of charge is no parameter here.
- They could improve their accessibility in creating forms that are not
crafted, that can be open by any program designed to open the standard.
- we can help them to figure out how to do that
Yes, we need to explain to them (I only mailed with one individual there, so I'm not sure how the rest of the Police understands this matter) the difference between using "PDF" and "PDF that requires exclusively AA Reader" for file formats.
*we should not say* :
- The way they create their form is wrong -> they would probably react
not be very listening if we start telling them they are wrong, we should rather tell them they can be better.
We could start by complementing them on the choice of PDF before other formats around. Then inform them that PDFs can be different depending on what you use to create them with. Additionally we could say that we assume that their aim is to be open and accessible and that they can be even more so, should they create PDFs in a way that makes them more universally editable.
*What we need to search in preparation*:
- Free Software PDF readers are not very performant with forms. They can
usually open them, but not save the information entered, and not perform a check-up routine. We need to determine which software are good at *editing* PDFs that can be open by all readers. It would be better here to have Free Software to suggest, in case they aren't, I guess it would already be an improvement to have institutions publish forms that can be open by Free Software even if they are created by a proprietary one...
- Rikard suggested there were possibilities to create forms with Open
Office or libre Office. Nico here tested Open Office, it worked for what concerns the form functions, but we couldn't figure out how to design the layout we wanted for the PDF... Some research here would be good before recommending these options
- other options would be good.
Stian perhaps has some input here?
- @Rikard : there is the possibility to create a wiki page on FSFE's
wiki (wiki.fsfe.org). If you don't have an account there, either one of the Fellows in Sweden could help you create the list of institutions, or you can create a guest account [2].
I'll check that out, thanks! If anyone beats me to it, in creating a sub page for this subset of the bugs involving PDF use, please let me know the URL to it!
*Do you have other ideas, comments on how we can procede here*?
I think putting together a guide on how to provide Forms that can be edited by other software than AAR is a great step.
Is using editable documents really a good idea for forms at all, btw? Woudn't it be better to have forms as part of their website? Then they could focus on following web standards in order to make the site as a whole accessible... Maybe I'm old-fashioned but I kind of like to print out forms and fill them in using a pen ;-) And I'm cynical enough to advice against interactive forms (having fields and tab order and input validation) as a file format for *text* since I'm convinced it will create interoperability and compatibility problems when accessed using different "readers" or even different versions of the same reader.
The fact we refer to them as "readers" implies somehow that they are primarily meant to read stuff and not for editing and saving stuff ;-).
But I'm not sure at all that this is relevant here. I just wanted to share my thoughts, because actually I reacted to the fact that you could download a "form" meant to be filled in using a computer. It may have security implications as well as the compatibility issues, I thought.
What do you all think about this? Should forms be part of a portable document format? If so, why and how should this be implemented?
Thanks for your involvement here,
My pleasure! Let's keep keeping in touch :)
Regards
Maëlle
Regards
Rikard
Hi Rikard and all, Since Maëlle left, I will be more involved in (indeed, in charge of) the pdfreaders campaign.
On 23/01/2011 22:48, Rikard Fröberg wrote:
Yes, we need to explain to them (I only mailed with one individual there, so I'm not sure how the rest of the Police understands this matter) the difference between using "PDF" and "PDF that requires exclusively AA Reader" for file formats.
Good way to go.
*we should not say* :
- The way they create their form is wrong -> they would probably react
not be very listening if we start telling them they are wrong, we should rather tell them they can be better.
We could start by complementing them on the choice of PDF before other formats around. Then inform them that PDFs can be different depending on what you use to create them with. Additionally we could say that we assume that their aim is to be open and accessible and that they can be even more so, should they create PDFs in a way that makes them more universally editable.
Completely agree.
*What we need to search in preparation*:
- Free Software PDF readers are not very performant with forms. They can
usually open them, but not save the information entered, and not perform a check-up routine. We need to determine which software are good at *editing* PDFs that can be open by all readers. It would be better here to have Free Software to suggest, in case they aren't, I guess it would already be an improvement to have institutions publish forms that can be open by Free Software even if they are created by a proprietary one...
- Rikard suggested there were possibilities to create forms with Open
Office or libre Office. Nico here tested Open Office, it worked for what concerns the form functions, but we couldn't figure out how to design the layout we wanted for the PDF... Some research here would be good before recommending these options
- other options would be good.
Stian perhaps has some input here?
*ping*
- @Rikard : there is the possibility to create a wiki page on FSFE's
wiki (wiki.fsfe.org). If you don't have an account there, either one of the Fellows in Sweden could help you create the list of institutions, or you can create a guest account [2].
I'll check that out, thanks! If anyone beats me to it, in creating a sub page for this subset of the bugs involving PDF use, please let me know the URL to it!
Have you got to have an account in the meantime? I would not like to miss your input on the subject, since you've demonstrated to have a broad knowledge and deep involvment with the pdf forms matter. I believe that this page can become very good if you still would like to take care of writing it (you can send me plain-text information too and I'll format/upload them, let's not be stopped by technical futilities).
*Do you have other ideas, comments on how we can procede here*?
I think putting together a guide on how to provide Forms that can be edited by other software than AAR is a great step.
Maybe the same wikipage can become such a guide
Is using editable documents really a good idea for forms at all, btw? Woudn't it be better to have forms as part of their website? Then they could focus on following web standards in order to make the site as a whole accessible... Maybe I'm old-fashioned but I kind of like to print out forms and fill them in using a pen ;-) And I'm cynical enough to advice against interactive forms (having fields and tab order and input validation) as a file format for *text* since I'm convinced it will create interoperability and compatibility problems when accessed using different "readers" or even different versions of the same reader.
I'm not sure it is time-worthy to enter in such a discussion with institutions. In my opinion they'll just stuck with their current process if we don't bring them an easy means to do it better. But I agree with you that pdf forms is a loose form of process automation… Web forms would work far better.
By the way, I propose we don't talk about "editing" but about "filling", since we don't edit the structure or texts inside the pdf, just fill in the blanks (or do I miss something here?).
But I'm not sure at all that this is relevant here. I just wanted to share my thoughts, because actually I reacted to the fact that you could download a "form" meant to be filled in using a computer. It may have security implications as well as the compatibility issues, I thought.
What do you all think about this? Should forms be part of a portable document format? If so, why and how should this be implemented?
I don't think so; however, as said above, the security implications or compatibility problems are not strong enough (they will believe) to have institutions change to more complex web-based forms. So I propose we stick with helping them improve their pdf forms.
This morning I generated a pdf form using open office writer, open it with evince, filled the form, ticked a tickbox, saved the whole, reopened it with evince: everything looks good at all steps of the process. The layout problems Maëlle mentioned previously are gone. I'm attaching the documents.