java.lang.IllegalArgumentException: Unable to use KSS for Demo Key Stores, KSS is unavailable..

Version:
======

WebLogic Server Version: 12.2.1.0.0
Enterprise Linux 6


Issue
=====

Weblogic domain "AdminServer" was not starting up due to the error mentioned in the post heading.


Analysis
=======

Admin server.out file had the following error stack.

AdminServer.out
---------------------
Could not load Logmanager "oracle.core.ojdl.logging.ODLLogManager"
java.lang.ClassNotFoundException: oracle.core.ojdl.logging.ODLLogManager
        at com.oracle.classloader.PolicyClassLoader.loadClass(PolicyClassLoader.java:267)
        at com.oracle.classloader.weblogic.LaunchClassLoader.loadClass(LaunchClassLoader.java:62)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.util.logging.LogManager$1.run(LogManager.java:195)
        at java.util.logging.LogManager$1.run(LogManager.java:181)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.util.logging.LogManager.<clinit>(LogManager.java:181)
        at java.util.logging.Handler.<init>(Handler.java:50)
        at java.util.logging.StreamHandler.<init>(StreamHandler.java:118)
        at com.oracle.classloader.log.LogStreamHandler.<init>(LogStreamHandler.java:58)
        at com.oracle.classloader.log.LogStreamHandler.<init>(LogStreamHandler.java:47)
        at com.oracle.classloader.log.LogStreamHandler.<init>(LogStreamHandler.java:30)
        at com.oracle.classloader.log.Logger.getLogHandler(Logger.java:135)
        at com.oracle.classloader.log.Logger.access$100(Logger.java:25)
        at com.oracle.classloader.log.Logger$BootstrapHandler.initialize(Logger.java:100)
        at com.oracle.classloader.log.Logger.initialize(Logger.java:71)
        at com.oracle.classloader.PolicyClassLoader.initializeLogging(PolicyClassLoader.java:82)
        at com.oracle.classloader.weblogic.LaunchClassLoader.loadClass(LaunchClassLoader.java:65)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
<Jul 24, 2018 11:39:18 AM GMT> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>
<Jul 24, 2018 11:39:18 AM GMT> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.>
<Jul 24, 2018 11:39:18 AM GMT> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 25.51-b03 from Oracle Corporation.>
<Jul 24, 2018 11:39:18 AM GMT> <Info> <RCM> <BEA-2165021> <"ResourceManagement" is not enabled in this JVM. Enable "ResourceManagement" to use the WebLogic Server "Resource Consumption Management" feature. To enable "ResourceManagement", you must specify the following JVM options in the WebLogic Server instance in which the JVM runs: -XX:+UnlockCommercialFeatures -XX:+ResourceManagement.>
<Jul 24, 2018 11:39:18 AM GMT> <Info> <Management> <BEA-141107> <Version: WebLogic Server 12.2.1.0.0 Tue Oct  6 10:05:47 PDT 2015 1721936>
<Jul 24, 2018 11:39:19 AM GMT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: A MultiException has 9 exceptions.  They are:
1. weblogic.management.provider.internal.RuntimeAccessImpl$ParseException: [Management:141266]Parsing failure in config.xml: java.lang.IllegalArgumentException: Unable to use KSS for Demo Key Stores, KSS is unavailable..
2. java.lang.IllegalStateException: Unable to perform operation: create on weblogic.management.provider.internal.RuntimeAccessImpl
3. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.management.provider.internal.RuntimeAccessService


Reason:
=======

setDomainEnv.sh file was edited and an entries for a few additional JAVA_OPTIONS were added.
One particular line was ended with two apostrophes instead of one ("")
This made all the libraries missing when it was called.

Solution:
========

Removed the additional apostrophe and succesfully restarted services

[DeploymentService:290066]Error occurred while downloading files from admin server

Version
========

Weblogic 10.3.6
Enterprisse Linux 6

Error :
=======

While trying to deploy a file  and click save below given error was found



An error occurred during activation of changes, please see the log for details.
Message icon - Error Exception occured while downloading files
Message icon - Error [DeploymentService:290066]Error occurred while downloading files from admin server for deployment request "1,517,969,705,031". Underlying error is: "null"

Checking Admin Server Log
=====================

####<Feb 7, 2018 5:44:22 AM UTC> <Info> <Deployer> <mmvbaxaesgp0010.int.oracleindustry.com> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <85f6879dee15bd51:77664bc0:1616df7a653:-8000-000000000000008f> <1517982262166> <BEA-149038> <Initiating Task for Signal : [Deployer:149026]deploy application Signal on SignalServer.>
####<Feb 7, 2018 5:44:31 AM UTC> <Warning> <DeploymentService> <host1.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <85f6879dee15bd51:77664bc0:1616df7a653:-8000-0000000000000090> <1517982271238> <BEA-290065> <Deployment service servlet encountered an Exception while handling the deployment datatransfer message for request id "1,517,982,262,115" from server "SignalServer". Exception is: "java.io.FileNotFoundException: /u01/app/Oracle/Install/App/relnotes/Release_Notes.pdf (Permission denied)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)
        at weblogic.utils.jars.JarFileUtils.addFilesToJar(JarFileUtils.java:120)

Also so messages in log like

####<Feb 7, 2018 5:44:31 AM UTC> <Warning> <Deployer> <mmvbaxaesgp0010.int.oracleindustry.com> <AdminServer> <[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <85f6879dee15bd51:77664bc0:1616df7a653:-8000-0000000000000091> <1517982271450> <BEA-149078> <Stack trace for message 149004
weblogic.management.DeploymentException: Exception occured while downloading files

Reason
======
It was a case of improper permission settings at file level. While checking that particular file, it was identified that the ownership of the file was with root whereas the application owner was oracle.

Solution
======

Changed permission and solved

-rw-r-----. 1 root       root         188529 Nov 13 12:31 Release_Notes.pdf
 cd /u01/app/Oracle/Install/App/relnotes


Changed to
-rw-r-----. 1 oracle       oinstall        188529 Nov 13 12:31 Release_Notes.pdf