Register  Login       Search  
Forum  
 
  
 
Forum  Forum     
 
SearchForum Home
  XML  Employing XML in .NET  Loading a huge ...
 Loading a huge xml document throw System.OutOfMemoryException
 
darioromero
2 posts
Joined
3/1/2006

Loading a huge xml document throw System.OutOfMemoryException
Posted: 01 Mar 06 2:15 PM (Canada)
I am using nxslt on a 2GB RAM memory machine with .NET Framework 1.1 and when trying to load an xml file 500Mb size I get this message:

Error occurred while loading document 'output.xml'.

System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.


The xslt transformation is very simple. Pulling some data from creating multiple output files as text format.

If you can drive me in the right direction that would be really appreciated.

Thank you and congratulations for creating nxslt. That is great.

Dario
olegt
85 posts
www.xmllab.net
Joined
2/25/2005

Re: Loading a huge xml document throw System.OutOfMemoryException
Posted: 06 Mar 06 7:26 AM (Israel)

Usual approach is to avoid using XSLT on huge documents. XSLT by design requires the whole source document to be represented as in-memory tree...

I'd try writing XmlReader-based code instead of using XSLT.


Oleg Tkachenko, Microsoft MVP for XML, MCPD
http://www.XmlLab.Net | http://blog.tkachenko.com
darioromero
2 posts
Joined
3/1/2006

Re: Loading a huge xml document throw System.OutOfMemoryException
Posted: 06 Mar 06 4:08 PM (Canada)
Thank you for replying back so soon.

Do you known of a good XmlReader-based code where I can also use nxslt ?

Thank you again
olegt
85 posts
www.xmllab.net
Joined
2/25/2005

Re: Loading a huge xml document throw System.OutOfMemoryException
Posted: 11 Apr 06 1:05 PM (N/A)

Sorry for late answer.

No, XmlReader approach assumes no XSLT is involved. XSLT requires full source document to be loaded into memory, which is you want to avoid. When using XmlReader you write code that reads XML node by node in a forward only non caching way - that's why it works great with huge XML documents, but obviously this model limits your processing - you can't pull arbitrary parts of XML - each node is visited only once.

Take a look at:

1. "Efficient Techniques for Modifying Large XML Files"

2. "Combining the XmlReader and XmlWriter classes for simple streaming transformations"

 


Oleg Tkachenko, Microsoft MVP for XML, MCPD
http://www.XmlLab.Net | http://blog.tkachenko.com
  XML  Employing XML in .NET  Loading a huge ...
Forum Home  Search