Too many open files in system error – caused by MyEclipse-2017

This topic contains 19 replies, has 3 voices, and was last updated by Profile photo of Sven Schmidt Sven Schmidt 23 minutes ago.

Viewing 15 posts - 1 through 15 (of 20 total)
  • Author
    Posts
  • #537879 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Recently I’ve updated MyEclipse on Mac to ME-2017. My project versioning is done with git. Therefore I’m using the integrated git for team development. Last week suddenly I got the error Too many open files in system error while running some terminal command. After reboot I wrote a script to find the offending process if this happens again – this script pointed to MyEclipse.

    $ lsof | awk ‘{print $2}’ | uniq -c | sort -rn | head | while read count pid ; do echo -n ‘Anzahl:’ ${count} ‘PID:’ ${pid} ‘Command:’ ; ps -fp ${pid} | tail +2 | awk ‘{ print $8 ; }’ ; done
    Anzahl: 9159 PID: 23530 Command:/Applications/MyEclipse
    Anzahl: 559 PID: 377 Command:/Applications/SourceTree.app/Contents/MacOS/Sourcetree
    Anzahl: 476 PID: 48800 Command:/Applications/MyEclipse
    Anzahl: 394 PID: 379 Command:/Applications/Xcode.app/Contents/MacOS/Xcode
    Anzahl: 353 PID: 2607 Command:/Applications/Mail.app/Contents/MacOS/Mail
    Anzahl: 305 PID: 56306 Command:/Users/Shared/LyX/lyx-build/LyX-2.4.0dev.app/Contents/MacOS/lyx
    Anzahl: 251 PID: 388 Command:/System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
    Anzahl: 239 PID: 469 Command:/System/Library/CoreServices/Spotlight.app/Contents/MacOS/Spotlight
    Anzahl: 171 PID: 59410 Command:/System/Library/StagedFrameworks/Safari/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc/Contents/MacOS/com.apple.WebKit.WebContent
    Anzahl: 152 PID: 2689 Command:/Applications/Safari.app/Contents/MacOS/Safari

    I think it’s a bad interaction of the git code in MyEclipse with Sourcetree – another git client.

    What can be done to avoid this?

    Best regards,
    Stephan

    #537939 Reply
    Profile photo of Brian Fernandes
    Brian Fernandes
    Moderator

    Stephan,

    The number of files open would depend on the projects you have open in your workspace, and the number of files therein. Whether you are actively working on a project would make a difference too. The bottom line is that this is not something we can easily control, and is typically affected more by the underlying JVM and OS behaviour/settings.

    The one change in 2017 is that we added Angular project support, if you have added Angular projects to your workspace, this could result in a large number of files being accessed (thanks to Node dependencies) though not sure this would be your problem.

    Do you have a large workspace, with a large number of resources? Any mods to your workspace since your previous version of MyEclipse (which version were you using prior to this)?

    Could you also tell me exactly which version of MyEclipse you are using now? Help > About will show you a build Id.

    The simplest solution I can propose would be to increase the file limit for your system – personally we have to do this in other Eclipse (not MyEclipse) environments too, when dealing with large workspaces.

    Hope this helps.

    #537991 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Hi Brian,

    I don’t know what you call a large workspace… but I’d guess the correct answer is yes (6844 files, 478 Java sources). I’m not sure if I’m using Angular project support. I did not change the structure of the project, but I’ve created a new workspace for the new MyEclipse environment.

    Previous Version was ME-2010 something (I’ve removed it already).

    Current MyEclipse Enterprise Workbench
    Version: 2017 Stable 1.0
    Build id: 15.0.0-20170329

    Your proposed solution is acceptable as a stopgap against dataloss in other applications. I’ve changed these settings already. I raised the system limit and the per process limit should stop MyEclipse from eating up the system resources completely. But I think MyEclipse is running into problems too if the maximum count of file descriptors is reached. The worst thing what might happen is some damage to a essential file in the MyEclipse internals.

    IMO there is a file descriptor leak in git locking or something similar. I’ll attach a screen shot of the error message I got from SourceTree (running in parallel).

    Stephan

    Attachments:
    You must be logged in to view attached files.
    #538491 Reply
    Profile photo of support-swapna
    support-swapna
    Moderator

    Stephan,

    Thank you for the information and the screenshot. We will investigate further and get back to you.

    –Swapna
    MyEclipse Support

    #538763 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Hi Swapna,

    please find attached the error message popup of MyEclipse when running for a longer time and reaching approximately 6000+ open file descriptors. The list of open file descriptors I’ve attached too. Now I don’t think anymore it is a git related problem.

    Stephan

    Attachments:
    You must be logged in to view attached files.
    #538765 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    myeclipse-open-ports.lst was rejected – trying it with myeclipse-open-ports.txt

    Attachments:
    You must be logged in to view attached files.
    #538854 Reply
    Profile photo of support-swapna
    support-swapna
    Moderator

    Sven,

    Thank you for the files. The dev team is looking into the problem. They will get back to you soon.
    Apologies for inconvenience caused.

    –Swapna
    MyEclipse Support

    #539165 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Hi Swapna,

    a co-worker had the same problem on Linux now and as a consequence he lost his workspace.

    Stephan

    #539178 Reply
    Profile photo of Brian Fernandes
    Brian Fernandes
    Moderator

    Stephan,

    I’m sorry to hear about the loss of the workspace – I hope you did not lose data, though I realize the workspace will need to be set up again. I’m assuming what happened is that workspace metadata would be corrupted, making it unusable by Eclipse, though projects in that workspace (if they existed in the same folder) should be fine.

    Now, coming to the issue, what clearly stands out in your log file is the large number of
    myeclipse 34494 stephan 989u IPv6 0x2955b74041948f43 0t0 UDP *:55979 entries.
    Given this entry, and the fact that your colleague ran into this too, on a different OS, perhaps this is related to your network setup – can you think of anything unique/odd about it?

    As far as tracking down this problem, are there any particular activities around which you see this problem? For example, is it typically during a checkout – or is it after you’ve deployed an app to the a server? Would it happen in sessions where you hadn’t used Git?

    If you can examine the number of open files before running into the too many files issue (using the same command you used to create the list) or when you run into the issue again, can you take a thread dump so we can see what is going on inside the IDE? To take a threaddump, please run
    jstack -l [PID] > dump.txt
    jstack is a binary found in the bin folder of your Java installation, replace [PID] with the process Id of MyEclipse, of course. SEnd us the resulting dump.txt file.

    Hope we’ll be able to get to the bottom of this with your input.

    #539193 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Brian,

    here you are.

    Stephan

    Attachments:
    You must be logged in to view attached files.
    #539229 Reply
    Profile photo of Brian Fernandes
    Brian Fernandes
    Moderator

    Stephan,

    Thank you for the files, I believe we have a fairly good idea of what is going on, and will keep you updated on a fix.

    Have you had MyEclipse continue to fail since your report on the 5th? Also, is your entire team using 2017 Stable 1.0? I ask because besides providing you/your team with a private fix, we will be releasing the fix publicly in the CI stream before it hits the Stable stream.

    #539437 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Brian,

    I hadn’t the reported error anymore since the 5th.

    Our team is using 2017 Stable 1.0 only on platforms Linux, Mac and Windows.

    #539784 Reply
    Profile photo of Brian Fernandes
    Brian Fernandes
    Moderator

    Stephan,

    In delivering the fix out of a regular release cycle, I’d like to discuss the mode of delivery.

    The simplest possible technique would be to give you a drop-in JAR file that would replace an existing file of the same name in your installations plugins folder.

    The other alternative would be to give you an archived update site with a fix, where you would have to add the site to your installation and apply the update for the fix to be applied.

    Would you prefer any particular solution?

    #540188 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Brian,

    we can choose the simple solution. That’s our preference.

    I hope this will be part of the official next stable release too, right? If not we’d like to know how to handle this.

    Stephan

    #544076 Reply
    Profile photo of Sven Schmidt
    Sven Schmidt
    Participant

    Brian,

    I have the feeling, I’ve lost you.

    In your last post you asked for a way to provide us with a fix out of the regular release cycle. We’re waiting for a solution. The problem is real for us.

    Stephan

Viewing 15 posts - 1 through 15 (of 20 total)
Reply To: Too many open files in system error – caused by MyEclipse-2017