MarkLogic 9에서 오류가 발생했습니다. 배터를 사용하는 Java 클라이언트

Question

엄청난 양의 데이터를로드하는 데 MarkLogic Java 클라이언트를 사용하고 있습니다. 잠시 후 다음과 같은 오류가 나타나고 그냥 반복됩니다.

23-May-2017 15:09:11.199 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [easymetahub] appears to have started a thread named [pool-20-thread-12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 
sun.misc.Unsafe.park(Native Method) 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) 
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
java.lang.Thread.run(Thread.java:745) 
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:105) 
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506) 
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) 
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) 
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) 
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) 
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) 
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1078) 
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) 
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) 
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1524) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
    at java.lang.Thread.run(Thread.java:745) 
java.lang.IllegalStateException: This instance has been stopped 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.requireNotStopped(WriteBatcherImpl.java:347) 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:283) 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:267) 
    at com.easymetahub.HarvestJDBCData.doRoot(HarvestJDBCData.java:486) 
    at com.easymetahub.HarvestJDBCData.doSomething(HarvestJDBCData.java:243) 
    at com.easymetahub.HarvestJDBCData.processBatchSegment(HarvestJDBCData.java:146) 
    at com.easymetahub.HarvestJDBCDataServlet.doPost(HarvestJDBCDataServlet.java:33) 
    at com.easymetahub.HarvestJDBCDataServlet.doGet(HarvestJDBCDataServlet.java:45) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) 
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:105) 
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506) 
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) 
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) 
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) 
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) 
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) 
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1078) 
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) 
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) 
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1524) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
    at java.lang.Thread.run(Thread.java:745) 
java.lang.IllegalStateException: This instance has been stopped 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.requireNotStopped(WriteBatcherImpl.java:347) 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:283) 
    at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:267) 
    at com.easymetahub.HarvestJDBCData.doRoot(HarvestJDBCData.java:486) 
    at com.easymetahub.HarvestJDBCData.doSomething(HarvestJDBCData.java:243) 
    at com.easymetahub.HarvestJDBCData.processBatchSegment(HarvestJDBCData.java:146) 
    at com.easymetahub.HarvestJDBCDataServlet.doPost(HarvestJDBCDataServlet.java:33) 
    at com.easymetahub.HarvestJDBCDataServlet.doGet(HarvestJDBCDataServlet.java:45) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232) 

나는이 내가 복사의 예에서 수정 한 다음 코드 : 해봐요 내가이 오류를 방지하려면 어떻게

   batcher.add(entityPath, getStringFromDocument(outDoc)); 

를 호출

 client = DatabaseClientFactory.newClient("localhost", 8000, "emh-entity-manager-content", new DatabaseClientFactory.DigestAuthContext("admin", "mladmin")); 
    DataMovementManager dmm = client.newDataMovementManager(); 
    batcher = dmm.newWriteBatcher(); 
    batcher.withBatchSize(5000) 
      .withThreadCount(30) 
      .onBatchSuccess(batch-> { 
       System.out.println(
         batch.getTimestamp().getTime() + 
           " documents written: " + 
           batch.getJobWritesSoFar() + 
           " \t[" + sourceName + "] [" + start + "]"); 
      }) 
      .onBatchFailure((batch,throwable) -> { 
       System.out.println("Failure on " + sourceName); 
       throwable.printStackTrace(); 
      }); 

    dmm.startJob(batcher); 

    doSomething(harvestNode); 
    // Start any partial batches waiting for more input, then wait 
    // for all batches to complete. This call will block. 
    batcher.flushAndWait(); 
    dmm.stopJob(batcher); 

?

Answer 1

코드 샘플에서 잘못된 것을 보지 못했습니다. 내가 공유 한 스택 추적이 다른 코드에서 온 것 같아요. 일반적으로 HarvestJDBCDataServlet.doPost와 같은 메서드 안에 새 DatabaseClient를 만들지 않으려는 경우 DatabaseClient 인스턴스에 공유 연결 풀이 포함되어 있기 때문에 WebClient의 수명 동안 DatabaseClient 인스턴스를 공유해야하기 때문입니다. dmm.stopJob (batcher)을 이미 호출 한 후에 batcher.add를 호출하면 오류가 표시됩니다. 이것은 로그에서 유일한 오류입니까, 아니면 다른 오류가 있습니까? 어떤 경우에도 버그가 발생한 것으로 생각되면 github에 문제를 신고하십시오.