2010-05-12

Troubleshooting PT 8.48 PT 8.49 Integration Broker

http://iambic.homeip.net/site/doku.php/tech/psoft/ib_troubleshooting
====== Troubleshooting PT 8.48 PT 8.49 Integration Broker ======
Troubleshooting guidance for publication process, subscription process, and other possible problems.

===== Publication Process Problems =====
 Possible publication process problems include:\\
  * Publication contract is not created.
  * Publication contract is in NEW status.
  * Publication contract stays in RETRY status.
  * Publication contract is in WORKING status.
  * Publication contract is in TIMEOUT status.
  * Publication contract is in ERROR status.

==== Pub Contract Is Not Created ====
  * No publication PeopleCode exists.
  * Publication PeopleCode is incorrect.
  * No outbound routings exists for the service operation.

==== Pub Contract Is in NEW Status ====
  * Sending queue is paused.
  * Publication Dispatcher crashed or was brought down.
  * Sending node is paused.
  * Previous service operation had a status of Retry, Error, or Timeout.
  * Sending domain is not active.
  * After an unsuccessful ping, a row may be added to the table PSNODESDOWN; which will hold up the message queue; query the table PSNODESDOWN.

==== Pub Contract Stays in RETRY Status ====
  * The remote node cannot be pinged successfully; the publication contract will be processed when the remote node comes back up.
  * No publication handler is available, either because it crashed or it was brought down.
  * Receiving Node URL is incorrect in integrationGateway.properties file.

==== Pub Contract Is in WORKING Status ====
  * The publication handler processing the contract is on another machine and either the machine or the domain is down. Processing should continue when the pub/sub system on the other machine comes back up.
  * Single threading on the application server is slowing processing.

==== Pub Contract Is in TIMEOUT Status ====
  * An exception occurred on the target application server (look in APPSRV.LOG file for details); verify that:
  * The reply is incorrectly routed; check Gateway for correct machine address of target node.
  * Bad XML syntax.

==== Pub Contract Is in ERROR Status ====
  * Receiving node user profile not authorized to service operation.
  * Inbound routing is not set up on the receiving system.
  * Service operation is not active on the receiving system.
  * Service operation has not been granted security access on the receiving system.
  * The source node is not defined in the target database.
  * Handler PeopleCode is bad.
  * Remote application server is down.
  * Receiving Node is not defined in the integrationGateway.properties file.
  * Receiving PeopleSoft node has not been added to single signon.
  * Service operation version on target is not active.

===== Subscription Process Problems =====

Possible subscription process problems include:\\
  * Subscription contract is not created.
  * Subscription contract is in NEW status.
  * Subscription contract is in STARTED status.
  * Subscription contract is in WORKING status.
  * Subscription contract is in ERROR status.
  * Subscription contract is in EDIT status.

==== Sub Contract Is not Created ====
  * No handler exists for the service operation.
  * Service operation handler is missing method.
  * Queue routing rules not set up properly.

==== Sub Contract Is in NEW Status ====
  * Application Server down.
  * Pub/Sub processes not configured on Application Server domain.
  * The Subscription Dispatcher has crashed or has been brought down.
  * Receiving queue is paused.
  * Receiving node is paused.
  * Previous service operation had errors or timed out.
  * No row was inserted into PSAPMSGSUBPRCID, To insert a row enter the following SQL statement in your Query tool use the following statement:
insert into PSAPMSGSUBPRCID values(0)

==== Sub Contract Is in STARTED Status ====
  * Subscription Handler is down.
  * Target component is not valid.

==== Sub Contract Is in WORKING Status ====
  * Subscription Handler crashed while processing message.

==== Sub Contract Is in ERROR Status ====
  * Queue property if Ordered enables subscription contracts to go in random order, which causes FULLSYNC service operations to error out  when the transaction is subscribed before the header.
  * Service operation handler PeopleCode errors exist.
  * Application data errors exist.

==== Sub Contract Is in EDIT Status ====
Possible cause is that the XML was edited and is not yet resubmitted for processing.

===== Other Possible Problems =====
Other possible problems include:\\
  * Cannot find service operation in Service Operation Monitor.
  * Service operations are being processed in an incorrect order.
  * Service operation is not created.
  * Service operation instance stays in NEW status.
  * Service operation instance stays in STARTED status.
  * Service operation instance stays in WORKING status.
  * Unable to ping a node.
  * Queue is PAUSED.

==== Cannot Find Service Operation in Service Operation Monitor ====
  * Possible causes is that filtering is set in the Service Operation Monitor.

==== Service Operations are Being Processed in an Incorrect Order ====
  * Possible cause is that the queue was partitioned and the resulting subqueues do not match what was assumed for the ordering of the service operations.

==== Service Operation Instance not Created ====
  * Possible cause is that the service operation is inactive.

==== Service Operation Instance Stays in NEW Status ====
  * The Application server is down.
  * Pub/sub services are not configured on the Application Server domain.
  * The Message Dispatcher crashed or was brought down.
  * The item is not at the top of the queue; all service operations with the same queue or subqueue are in the same queue.

==== Service Operation Instance Stays in STARTED Status ====
  *  All Message Handlers crashed or were brought down; processing will resume when Message Handlers come back up.
  * The Message dispatcher processing the message is on another machine, and either the machine or the application  server domain is down.

==== Service Operation Instance Stays in WORKING Status ====
  * Message Broker Handler crashed.
  * The Message Handler processing the message is on another machine, and either the machine or the application server domain is down.
  * The Message Handler working on the message is blocked. The service will time out, and the Message Dispatcher will retry the message.

==== Unable to Ping a Node ====
  * The web server for the Gateway is down.
  * The Gateway is not configured properly.
  * The application server for the node is down.
  * The Gateway URL is incorrect; verify that the URL is correct.
  * Copy URL in browser address; you should see: \\
   
    PeopleSoft Integration Gateway
    PeopleSoft Listening Connector
    Status: ACTIVE
   

==== Queue Is PAUSED ====
Possible cause is that some queues are delivered as paused. Change the status to Run for the service operations to process from New to Working.

===== Unable to find a Routing corresponding to the incoming request message =====
You are sending an async service operation from node A to target node B. The nodes are pingable.  The routing on target database and source database exist and are correct.  The same service operation works fine if you use and "ANY TO LOCAL" routing.

The "ANY TO LOCAL" routing type uses a virtual node ANY when node passwords fail and eliminates node authentication from the equation.  If "ANY TO LOCAL" works, but a standard point to point routing fails, reset the node authentication option to NONE rather than password and try again. If this works, then the node passwords were incorrect. Resetting these should resolve the problem.

Note: You can also see this error if the Message name does not match the //externalservice// operation name. **They have to Match.**

===== Unable to ping nodes. "Issuing node is not a trusted node" or "PSFT Authentication token failed for Node PSFT_HR" =====
Application server log shows:

PSAPPSRV.6475 (32) [10/17/06 22:09:45 VP1@JavaClient IntegrationSvc](3) PeopleSoft Token authentication failed: issuing node PSFT_HR is not a trusted node : PS@JavaClient

Starting with Peopletools 8.48 Integration Broker requires that nodes be //trusted// on both sides before pinging or sending messages.
To fix this, navigate to  Peopletools --> Security Objects --> Single Signon --> "Trust Authentication Tokens issued by these Nodes"  and add the appropriate node names on each side.

2 comments:

Udaya said...

Hi,
Thank you for the post which is very useful.

Can you please expand the cause:

"Queue routing rules not set up properly."

for "Sub Contract Is not Created" issue.

Thanks in advance.

UB.

Alexei Tetenov said...

Another entry for the "Sub Contract Is not Created" category.
Sometimes after a refresh, you need to inactive and re-activate the routings.