首页 \ 问答 \ 为具有瞬态scala依赖性的spring-xd项目优化gradle(Optimize gradle for spring-xd project with transient scala dependency)

为具有瞬态scala依赖性的spring-xd项目优化gradle(Optimize gradle for spring-xd project with transient scala dependency)

我正在开发一个项目,其中包含一系列Spring XD的模块子项目,这些子项目碰巧对碰巧使用Scala的非模块子项目具有瞬态依赖性:

ext {
  springXdVersion = '1.1.0.RELEASE'
  moduleProjects  = subprojects.findAll { project -> project.path.startsWith(':modules.')}
  javaProjects    = subprojects - (moduleProjects + nonJavaProjects)
}

configure(moduleProjects) { moduleProject ->
  apply plugin: 'spring-xd-module'
}

project('core-dependency') {
  apply plugin: 'scala'
  // configuration/dependencies
}

project('modules.source.example') {
  dependencies {
    provided(":core-dependency")
  }
}

// More modules bearing resemblance to modules.source.example

核心依赖关系最终设置在xd-container的类路径中,并以这种方式在运行时提供给模块。

不幸的是,似乎对于使用它的每个模块,核心依赖性都会被重新编译(由于它也包含了scala编译,因此特别昂贵)。 这导致构建在30分钟以后运行,我想改进。 有没有办法缩短构建时间? 理想情况下,我不想重新编译核心依赖,但我不确定如何实现这一点,考虑到bootRepackage似乎负责为每个模块触发它。 我也尝试过其他技巧,例如并行性,但到目前为止,这样做只能冻结我的系统。 我正在使用gradle 2.1。

我应该注意,gradle配置文件报告指出,对于每个模块,大部分时间沉没都在configureModule步骤中,根据spring-xd repo,它看起来像这样:

project.task('configureModule') << {
            project.configurations.provided.resolvedConfiguration.firstLevelModuleDependencies.each {
                excludeTransitiveDependencies(project, it)
            }
        }

I am working on a project that contains a series of module subprojects for spring XD which happen to have a transient dependency on a non-module subproject that happens to use Scala:

ext {
  springXdVersion = '1.1.0.RELEASE'
  moduleProjects  = subprojects.findAll { project -> project.path.startsWith(':modules.')}
  javaProjects    = subprojects - (moduleProjects + nonJavaProjects)
}

configure(moduleProjects) { moduleProject ->
  apply plugin: 'spring-xd-module'
}

project('core-dependency') {
  apply plugin: 'scala'
  // configuration/dependencies
}

project('modules.source.example') {
  dependencies {
    provided(":core-dependency")
  }
}

// More modules bearing resemblance to modules.source.example

Core-dependency is ultimately set up to be in the classpath of our xd-container, and is provided to the modules at runtime in this manner.

Unfortunately, it seems that for each module that uses it, the core-dependency gets recompiled (which is particularly expensive since it includes a scala compile, too). This causes builds to run north of 30 minutes, which I'd like to improve upon. Is there a way to bring down the build time? Ideally, I'd like to not have to recompile core-dependency, but I'm not sure how to approach accomplishing this, considering that bootRepackage seems to be responsible for triggering it for each module. I have also tried other tricks, such as parallelism, but doing this has only managed to freeze my system so far. I am using gradle 2.1.

I should note that a gradle profile report indicates that for each module, most of the time sunk is in the configureModule step, which, according to the spring-xd repo, looks like this:

project.task('configureModule') << {
            project.configurations.provided.resolvedConfiguration.firstLevelModuleDependencies.each {
                excludeTransitiveDependencies(project, it)
            }
        }
更新时间:2023-03-21 16:03

最满意答案

scala依赖性来自SpringXD中的Spark streaming集成。 我们正在努力消除对spring-xd-dirt依赖性,并将其从模块中提供:

https://jira.spring.io/browse/XD-2857

您依赖的具体非模块子项目是什么? 如果它是spring-xd-module ,那么你可以尝试1.2.0.M1,我们已经将spring-xd-module的spark依赖项移动到了spring-xd-dirt


During the configureModule step, all projects are evaluated for transitive provided dependencies, which core-dependency was. The slowdown was caused by the sheer number of dependencies that core-dependency relies on, and consquently requires scanning. Since we want to avoid having core-dependency scanned by configureModule because it is too time-expensive, and we know it needs to be excluded from module fatjars, the appropriate course of action is to remove core-dependency from the provided configuration, and simply exclude it from the fat jar ourselves.

To accomplish this, the gradle build script is modified as follows:

ext {
  springXdVersion = '1.1.0.RELEASE'
  moduleProjects  = subprojects.findAll { project -> project.path.startsWith(':modules.')}
  javaProjects    = subprojects - (moduleProjects + nonJavaProjects)
}

configure(moduleProjects) { moduleProject ->
  apply plugin: 'spring-xd-module'

  configurations{
    core
    compile.extendsFrom(core)
  }

configurations.exported.exclude module: 'core-dependency'
}

project('core-dependency') {
  apply plugin: 'scala'
  // configuration/dependencies
}

project('modules.source.example') {
  dependencies {
    core project(":core-dependency")
  }
}

// More modules bearing resemblance to modules.source.example

The "core" configuration is essentially a second "provided" configuration, which will not get picked up by the configureModule task, avoiding the time wasting evaluation upon it. It is also excluded from the "exported" configuration, whose contents determines what goes into the fat jar that bootRepackage builds.

相关问答

更多

相关文章

更多

最新问答

更多
  • 在javascript中创建类以创建对象并在Java中创建类和对象之间的区别(Difference between creating a class in javascript to create an object and creating an class and object in Java)
  • Facebook API:将身份验证详细信息从Javascript SDK发送到PHP SDK(Facebook API: Send authentication detail from Javascript SDK to PHP SDK)
  • 如何停止队列动画jquery?(How can I stop queue animation jquery?)
  • 使用C#的井字游戏中的人工智能(Artificial Intelligence in Tic-Tac-Toe using C#)
  • 多少流量可以共享虚拟主机(对于Python Django站点)支持?(How Much Traffic Can Shared Web Hosting (for a Python Django site) support?)
  • 带有CIFilters的CAShapeLayer(CAShapeLayer with CIFilters)
  • 如何在Angular 2中读取JSON #text(How to read in Angular 2 the JSON #text)
  • 如何在xml中读取自闭标签的属性?(How to read self closing tag's attribute in xml?)
  • 无法使用http put将图像上传到亚马逊S3(Cannot upload image to amazon s3 using http put)
  • 文件结束无限循环(end of file infinite while-loop)
  • 在cpp的模板(template in cpp)
  • 在构建库时,clang和clang ++有什么区别?(What's the difference between clang and clang++ when building a library?)
  • ng类中的表达式(expression inside ng-class)
  • 在PHP中获取随机布尔值true / false(Get random boolean true/false in PHP)
  • 管道的高效分块用于严格的字节串(Efficient chunking of conduit for strict bytestring)
  • Python ternary_operator(如果其他标志做了其他操作,则执行其他操作)(Python ternary_operator (do-somthing if flag else do-another))
  • Sencha Touch面具发布(Sencha Touch mask ondisclosure)
  • 验证脚本上的通知[重复](Notices on validation script [duplicate])
  • 朋友功能(friend function)
  • 基于角坐标平移和变换平面几何(Translate and transform plane geometry based on corner coordinates)
  • Rails:'如果在本地运行'条件javascript标记包括(Rails: 'if running locally' conditional javascript tag include)
  • 解压文件(Unzipping files)
  • 使用ui-router以角度加载变量状态(loading in variable states with ui-router in angular)
  • 创建Azure云服务需要多长时间?(how long does it take to create an Azure Cloud Service? How to view log information?)
  • 指向整数的指针数组(Array of pointers to integers)
  • Laravel服务提供商没有看到我的包的主要类(Laravel service provider does not see the main class of my package)
  • 这个关于VSS / RSS / PSS / USS的解释是否准确?(Is this explanation about VSS/RSS/PSS/USS accurate?)
  • 在Django-Admin中通过row-id排序显示项目(Ordering the display items by row-id in Django-Admin)
  • 如何使用cythonize启用`--embed`?(How to enable `--embed` with cythonize?)
  • 用于将文本多行设置的Excel脚本(Excel script for ereasing text multiple rows)