JavaEar 专注于收集分享传播有价值的技术资料

Wcf System.InsufficientMemoryException. Failed to allocate a managed memory buffer of 536870912 bytes

I have a WCF service method that returns a big byte[] array around 2 mb in size. However when i run that method from the WcfTestClient, I get the following exception.

System.InsufficientMemoryException
Failed to allocate a managed memory buffer of 536870912 bytes. The amount of available memory may be low.

I have searched on google, and people seem to have this issue but I am struggling to find any solution and the actual cause of why is it trying to create such a huge buffer in the first place?

Below are my config settings. I have set the maxItemsInObjectGraph="2147483647" otherwise it was throwing exception about it. The rest is simple i guess.

Server

  <system.serviceModel>
    <services>
      <service behaviorConfiguration="Wcf.ServiceBehavior"
        name="Wcf.WcfService">
        <endpoint address="" binding="wsHttpBinding" contract="Wcf.IWcfService">
          <identity>
            <dns value="localhost" />
          </identity>
        </endpoint>
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="Wcf.ServiceBehavior">
          <serviceMetadata httpGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="true" />
          <dataContractSerializer maxItemsInObjectGraph="2147483647" />
        </behavior>
      </serviceBehaviors>
    </behaviors>

    <diagnostics>    
      <messageLogging maxMessagesToLog="30000"    
              logEntireMessage="true"    
              logMessagesAtServiceLevel="true"    
              logMalformedMessages="true"    
              logMessagesAtTransportLevel="true">    
      </messageLogging>    
    </diagnostics>
  </system.serviceModel>

WcfTestClient Config This is the default config file that is loaded with WcfTestClient

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <system.serviceModel>
        <bindings>
            <wsHttpBinding>
                <binding name="WSHttpBinding_IWcfService" closeTimeout="00:01:00"
                    openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
                    bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
                    maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
                    messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
                    allowCookies="false">
                    <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                        maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                    <reliableSession ordered="true" inactivityTimeout="00:10:00"
                        enabled="false" />
                    <security mode="Message">
                        <transport clientCredentialType="Windows" proxyCredentialType="None"
                            realm="" />
                        <message clientCredentialType="Windows" negotiateServiceCredential="true"
                            algorithmSuite="Default" />
                    </security>
                </binding>
            </wsHttpBinding>
        </bindings>
        <client>
            <endpoint address="http://localhost:4151/Service1.svc" binding="wsHttpBinding"
                bindingConfiguration="WSHttpBinding_IWcfService" contract="IWcfService"
                name="WSHttpBinding_IWcfService">
                <identity>
                    <dns value="localhost" />
                </identity>
            </endpoint>
        </client>
    </system.serviceModel>
</configuration>

Here is the exception Stack Trace

    <E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent">

  <System xmlns="http://schemas.microsoft.com/2004/06/windows/eventlog/system">

    <EventID>131075</EventID>
    <Type>3</Type>
    <SubType Name="Error">0</SubType>
    <Level>2</Level>
    <TimeCreated SystemTime="2010-06-29T09:18:23.9616589Z" />
    <Source Name="System.ServiceModel" />
    <Correlation ActivityID="{7870ff09-e268-4e9d-a692-389fd03db1aa}" />
    <Execution ProcessName="WebDev.WebServer20" ProcessID="4812"
    ThreadID="17" />
    <Channel />
    <Computer>PC-008915</Computer>
  </System>
  <ApplicationData>
    <TraceData>
      <DataItem>
        <TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord"
        Severity="Error">
          <TraceIdentifier>
          http://msdn.microsoft.com/en-GB/library/System.ServiceModel.Diagnostics.ThrowingException.aspx</TraceIdentifier>
          <Description>Throwing an exception.</Description>
          <AppDomain>5ab4f443-1-129222741421051645</AppDomain>
          <Exception>
            <ExceptionType>System.InsufficientMemoryException,
            mscorlib, Version=2.0.0.0, Culture=neutral,
            PublicKeyToken=b77a5c561934e089</ExceptionType>
            <Message>Failed to allocate a managed memory buffer of
            536870912 bytes. The amount of available memory may be
            low.</Message>
            <StackTrace>at
            System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32
            size) at
            System.ServiceModel.Channels.BufferManager.GCBufferManager.TakeBuffer(Int32
            bufferSize) at
            System.ServiceModel.Channels.BufferedOutputStream.AllocNextChunk(Int32
            minimumChunkSize) at
            System.ServiceModel.Channels.BufferedOutputStream.WriteCore(Byte[]
            buffer, Int32 offset, Int32 size) at
            System.ServiceModel.Channels.BufferedOutputStream.Write(Byte[]
            buffer, Int32 offset, Int32 size) at
            System.Xml.XmlStreamNodeWriter.FlushBuffer() at
            System.Xml.XmlStreamNodeWriter.GetBuffer(Int32 count,
            Int32&amp; offset) at
            System.Xml.XmlStreamNodeWriter.UnsafeWriteUTF8Chars(Char*
            chars, Int32 charCount) at
            System.Xml.XmlStreamNodeWriter.WriteUTF8Chars(String
            value) at
            System.Xml.XmlUTF8NodeWriter.WriteEndElement(String
            prefix, String localName) at
            System.Xml.XmlSigningNodeWriter.WriteEndElement(String
            prefix, String localName) at
            System.Xml.XmlBaseWriter.WriteEndElement() at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteString(XmlWriterDelegator
            xmlWriter, String value, XmlDictionaryString name,
            XmlDictionaryString ns) at
            WritePropertyDtoToXml(XmlWriterDelegator , Object ,
            XmlObjectSerializerWriteContext , ClassDataContract )
            at
            System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator
            xmlWriter, Object obj, XmlObjectSerializerWriteContext
            context) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            WriteArrayOfPropertyDtoToXml(XmlWriterDelegator ,
            Object , XmlObjectSerializerWriteContext ,
            CollectionDataContract ) at
            System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator
            xmlWriter, Object obj, XmlObjectSerializerWriteContext
            context) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            WriteEntityTypeDtoToXml(XmlWriterDelegator , Object ,
            XmlObjectSerializerWriteContext , ClassDataContract )
            at
            System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator
            xmlWriter, Object obj, XmlObjectSerializerWriteContext
            context) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            WriteNodeTypeDtoToXml(XmlWriterDelegator , Object ,
            XmlObjectSerializerWriteContext , ClassDataContract )
            at
            System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator
            xmlWriter, Object obj, XmlObjectSerializerWriteContext
            context) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator
            xmlWriter, Object obj, Boolean isDeclaredType, Boolean
            writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle
            declaredTypeHandle) at
            WriteArrayOfNodeTypeDtoToXml(XmlWriterDelegator ,
            Object , XmlObjectSerializerWriteContext ,
            CollectionDataContract ) at
            System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator
            xmlWriter, Object obj, XmlObjectSerializerWriteContext
            context) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
            dataContract, XmlWriterDelegator xmlWriter, Object obj,
            RuntimeTypeHandle declaredTypeHandle) at
            System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator
            writer, Object graph) at
            System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator
            writer, Object graph) at
            System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator
            writer, Object graph) at
            System.Runtime.Serialization.XmlObjectSerializer.WriteObject(XmlDictionaryWriter
            writer, Object graph) at
            System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeParameterPart(XmlDictionaryWriter
            writer, PartInfo part, Object graph) at
            System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeParameter(XmlDictionaryWriter
            writer, PartInfo part, Object graph) at
            System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeBody(XmlDictionaryWriter
            writer, MessageVersion version, String action,
            MessageDescription messageDescription, Object
            returnValue, Object[] parameters, Boolean isRequest) at
            System.ServiceModel.Dispatcher.OperationFormatter.SerializeBodyContents(XmlDictionaryWriter
            writer, MessageVersion version, Object[] parameters,
            Object returnValue, Boolean isRequest) at
            System.ServiceModel.Dispatcher.OperationFormatter.OperationFormatterMessage.OperationFormatterBodyWriter.OnWriteBodyContents(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Channels.BodyWriter.WriteBodyContents(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Channels.BodyWriterMessage.OnWriteBodyContents(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Channels.Message.WriteBodyContents(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Security.SecurityAppliedMessage.WriteBodyToSignThenEncryptWithFragments(Stream
            stream, Boolean includeComments, String[]
            inclusivePrefixes, EncryptedData encryptedData,
            SymmetricAlgorithm algorithm, XmlDictionaryWriter
            writer) at
            System.ServiceModel.Security.WSSecurityOneDotZeroSendSecurityHeader.ApplyBodySecurity(XmlDictionaryWriter
            writer, IPrefixGenerator prefixGenerator) at
            System.ServiceModel.Security.SecurityAppliedMessage.OnWriteMessage(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Channels.Message.WriteMessage(XmlDictionaryWriter
            writer) at
            System.ServiceModel.Channels.BufferedMessageWriter.WriteMessage(Message
            message, BufferManager bufferManager, Int32
            initialOffset, Int32 maxSizeQuota) at
            System.ServiceModel.Channels.TextMessageEncoderFactory.TextMessageEncoder.WriteMessage(Message
            message, Int32 maxMessageSize, BufferManager
            bufferManager, Int32 messageOffset) at
            System.ServiceModel.Channels.HttpOutput.SerializeBufferedMessage(Message
            message) at
            System.ServiceModel.Channels.HttpOutput.Send(TimeSpan
            timeout) at
            System.ServiceModel.Channels.HttpRequestContext.OnReply(Message
            message, TimeSpan timeout) at
            System.ServiceModel.Activation.HostedHttpContext.OnReply(Message
            message, TimeSpan timeout) at
            System.ServiceModel.Channels.RequestContextBase.Reply(Message
            message, TimeSpan timeout) at
            System.ServiceModel.Security.SecuritySessionServerSettings.SecuritySessionRequestContext.OnReply(Message
            message, TimeSpan timeout) at
            System.ServiceModel.Channels.RequestContextBase.Reply(Message
            message, TimeSpan timeout) at
            System.ServiceModel.Channels.RequestContextBase.Reply(Message
            message) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Reply(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessageCleanup(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc&amp;
            rpc) at
            System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean
            isOperationContextSet) at
            System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Dispatch(MessageRpc&amp;
            rpc, Boolean isOperationContextSet) at
            System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext
            request, Boolean cleanThread, OperationContext
            currentOperationContext) at
            System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext
            request, OperationContext currentOperationContext) at
            System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult
            result) at
            System.ServiceModel.Dispatcher.ChannelHandler.OnContinueAsyncReceive(Object
            state) at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke2()
            at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.OnSecurityContextCallback(Object
            o) at
            System.Security.SecurityContext.Run(SecurityContext
            securityContext, ContextCallback callback, Object
            state) at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke()
            at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ProcessCallbacks()
            at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.CompletionCallback(Object
            state) at
            System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ScheduledOverlapped.IOCallback(UInt32
            errorCode, UInt32 numBytes, NativeOverlapped*
            nativeOverlapped) at
            System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32
            error, UInt32 bytesRead, NativeOverlapped*
            nativeOverlapped) at
            System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32
            errorCode, UInt32 numBytes, NativeOverlapped*
            pOVERLAP)</StackTrace>
            <ExceptionString>System.InsufficientMemoryException:
            Failed to allocate a managed memory buffer of 536870912
            bytes. The amount of available memory may be low.
            ---&gt; System.OutOfMemoryException: Exception of type
            'System.OutOfMemoryException' was thrown. at
            System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32
            size) --- End of inner exception stack trace
            ---</ExceptionString>
            <InnerException>
              <ExceptionType>System.OutOfMemoryException, mscorlib,
              Version=2.0.0.0, Culture=neutral,
              PublicKeyToken=b77a5c561934e089</ExceptionType>
              <Message>Exception of type
              'System.OutOfMemoryException' was thrown.</Message>
              <StackTrace>at
              System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32
              size)</StackTrace>
              <ExceptionString>System.OutOfMemoryException:
              Exception of type 'System.OutOfMemoryException' was
              thrown. at
              System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32
              size)</ExceptionString>
            </InnerException>
          </Exception>
        </TraceRecord>
      </DataItem>
    </TraceData>
  </ApplicationData>
</E2ETraceEvent>

Awaiting Nabeel

3个回答

    最佳答案
  1. Are you sure your byte array is only around 2MB? From the exception message that you posted it seems like the message in the buffer is approaching 0.5GB (536870912 bytes). Is this error happening consistently? Are you running multiple service calls at the time this occurs?

    Also understand that when using WSHttpBinding with Message Security that the whole message is buffered. My understanding is that this is done to encrypt the whole message in one block so that the corresponding details can be supplied in the header. In other words the header needs the length of the message which is unknown until the encryption is done.

    See my related Question where I ask How to best transfer large payloads of data using wsHttp with WCF with message security.

  2. 参考答案2
  3. Our Application Pool was set to Recycle every 0 minutes (not recycle). After changing it back to 1740, and manually recycling the pool, the error went away. I think the Application Pool runs out of available / committed memory and returns that message if all your objects haven't been disposed properly in .NET

  4. 参考答案3
  5. We also had exactly the same problem. Tried to download a document of only 15mb and the memory usage sky rocketed to about 1gb. This resulted in an outofmemory exception on the server. The solution for us was very simple. The data contract implemented an ICollection for the ByteArray. When we changed this in to a Byte[] the problem was gone and the memory usage was normal again. Apparently the serialization process can not handle the ICollection type in an efficient way.