Thursday 15 June 2017

ERROR: Failed to check out svn project. vn: E170001: CONNECT of ‘domain’: 403 Forbidden



Today I got a 403 forbidden error when I was trying to checkout code from an SVN repository via Jenkins.

Checking out https://domain/svn/my/my_repos/myproject at revision '2017-06-15T12:11:23.590 +0530'
ERROR: Failed to check out https://domain/svn/my/my_repos/myproject
org.tmatesoft.svn.core.SVNAuthenticationException: svn: E175002: CONNECT request failed on 'http://proxyserver:3128'
svn: E170001: CONNECT of ‘domain’: 403 Forbidden (https://domain)
        at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:110)
        at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:96)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:263)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:454)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:391)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:379)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:862)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:698)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:118)
        at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1049)
        at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:189)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getRevisionNumber(SVNBasicDelegate.java:480)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getLocations(SVNBasicDelegate.java:833)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:527)
        at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doCheckout(SVNUpdateClient16.java:875)
        at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:66)
        at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:18)
        at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
        at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1235)
        at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
        at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:119)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:162)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:170)
        at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:134)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:162)
        at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:996)
        at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:972)
        at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:948)
        at hudson.FilePath.act(FilePath.java:997)
        at hudson.FilePath.act(FilePath.java:975)
        at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:897)
        at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:833)
        at hudson.scm.SCM.checkout(SCM.java:495)
        at hudson.model.AbstractProject.checkout(AbstractProject.java:1212)
        at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
        at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
        at hudson.model.Run.execute(Run.java:1735)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:415)
org.tmatesoft.svn.core.SVNAuthenticationException: svn: E175002: CONNECT request failed on 'http://proxyserver:3128'
svn: E170001: CONNECT of ‘domain’: 403 Forbidden (https://domain )
        at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:110)
        at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:96)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.connect(HTTPConnection.java:263)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:454)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:391)
        at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:379)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:862)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:698)
        at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:118)
        at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1049)
        at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:189)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getRevisionNumber(SVNBasicDelegate.java:480)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getLocations(SVNBasicDelegate.java:833)
        at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:527)
        at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doCheckout(SVNUpdateClient16.java:875)
        at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:66)
        at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:18)
        at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
        at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1235)
        at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
        at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:119)
Caused: java.io.IOException: Failed to check out https://domain/svn/my/my_repos/myproject
        at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:130)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:162)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:170)
        at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:134)
        at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:162)
        at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:996)
        at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:972)
        at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:948)
        at hudson.FilePath.act(FilePath.java:997)
        at hudson.FilePath.act(FilePath.java:975)
        at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:897)
        at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:833)
        at hudson.scm.SCM.checkout(SCM.java:495)
        at hudson.model.AbstractProject.checkout(AbstractProject.java:1212)
        at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
        at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
        at hudson.model.Run.execute(Run.java:1735)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:415)


I am using standard subversion plugin of Jenkins to checkout code.
Jenkins configuration is very simple Jenkins. I have just given the repository URL, global credentials and I have the left the default settings for remaining fields.
 
In the above screenshot, although Jenkins is complaining that it is unable to access repository, probably due to invalid credentials, credentials were correct. Even after issue got resolved, Jenkins still gives that error. So the root cause was something else.

After hours of debugging, I bothered to give a closer look at error message in build console. It was trying to go through a proxy. Now, the SVN server was on the company intranet, so there is no need of proxy. Quickly started to look around, where I have defined the proxy settings. I was running tomcat via eclipse. Had given –Dhttp.proxyHost=proxyserver –Dhttp.proxyPort=3128 in JVM arguments.

Note:
How to go to JVM arguments?
1.       Double click your server in Tomcat.
2.       Tomcat settings page opens up.
3.       In General Information section, you have Open Launch Configuration Link. Click it 
4.       Go to Arguments tab.
5.       In the VM arguments section, you can specify the arguments that you want to pass to the JVM which is running Tomcat.

So after removing proxy settings, it should have ideally worked. But it didn’t.
Later I figured out that catalina.properties was also having the proxy settings defined in it.

http.proxyHost=proxyserver
http.proxyPort=3128
https.proxyHost= proxyserver
https.proxyPort=3128

I did not want to comment them, as some other parts of Jenkins were using it. I was using same tomcat for other applications that needed the proxy settings. I needed some way to tell Jenkins to skip going through the proxy server for some hosts / domains. http.nonProxyHosts property to the rescue.

Added below 2 lines to catalina.properties. Multiple domains / hosts in http.nonProxyHosts property can be separated by |. People use “,” as separator, but that is wrong.

http.nonProxyHosts=localhost|*.domain.com
https.nonProxyHosts=localhost|*.aon.com

With that my checkout started working. Hope that helps someone.