JUnit 軟體測試架構 JUnit Software Testing Framework 葉秉哲 2002/7/4. http://william.cswiz.org/present/20020704 ● 講題介紹 無錯誤的軟體是軟體專案的夢想，但隨著軟體系統日益龐雜， 測試任務也越來越艱困。是否有既輕巧、威力強、彈性十足， 又經濟實惠的輔助工具呢？ Kent Beck 是 extreme programming 創始人， Erich Gamma 是赫赫有名的 Design Patterns 四巨頭之一， 這兩位軟體大師通力合作的產物：JUnit，究竟有什麼樣的魅力， 風靡軟體界，更於 2001～2002 連續兩年贏得 JavaWorld Editors’ Choice Awards 的 “Best Java Performance Monitoring/Testing Tool” 獎項呢？ 且讓我們一起來認識、掌握 JUnit 軟體測試架構吧！

JUnit 的定位 JUnit is… JUnit is not… 單元測試 整合測試 骨幹、架構 完整系統 工具 方法論 手動產生 scope 單元測試 整合測試 testware 骨幹、架構 完整系統 level 工具 方法論 test case 手動產生 自動產生 test driver / script 部份手寫 全自動 2

大綱 引子 JUnit 基礎篇 JUnit 進階篇 軟體測試基礎 推薦讀物 3

一、引子 一流的科學家可以做出很有價值的實驗，產生新知識；二流科學家只是忙於各種實驗，蒐集大量數據，但對知識的累積沒什麼用處。 --- David Salsburg

JUnit 簡介 Regression testing framework written by Erich Gamma and Kent Beck Open Source Hosted on SourceForge Language support Smalltalk, Java, C++, Perl, Python, etc. IDE Support JBuilder, VisualAge, etc. 5

What is Software Testing? The execution of code using combinations of input and state selected to reveal bugs. The process of devising and executing a test suite that attempts to cause failures, increasing our confidence that failures are unlikely. 6

Why Testing? 目的：矛與盾 重要性 Reveal bugs: quality control Confidence: qualitative & quantitative 重要性 Paradigm↑  productivity↑  testing & debugging↑ Software features that can’t be demonstrated by automated tests simply don’t exist. [Beck 1999, p.45] 7

Testing: CMM Perspective The Key Process Areas by Maturity Level Source: Capability Maturity Model for Software, v1.1 http://www.sei.cmu.edu/publications/documents/93.reports/93.tr.024.html 8

Testing: RUP Perspective Source: Philippe Kruchten, The Rational Unified Process: An Introduction, p.62, Addison Wesley, 1998. 9

Testing: XP Perspective Source: Kent Beck, Extreme Programming Explained: Embrace Change, p.70, Addison Wesley, 1999. If you’re not validating everything all the time, you’re not doing XP. Period. 10

Test What? Every artifact during the whole development process should be tested. 11

軟體測試工具分類 分類 實例 reviews & inspections test planning lint Compuware DevPartner test planning Rational Suite TestStudio test design & development test execution & evaluation capture/playback coverage analysis Borland Optimizeit memory testing test case management sumulations & performance profiler test support 12

JUnit 的定位 JUnit is… JUnit is not… 單元測試 整合測試 骨幹、架構 完整系統 工具 方法論 手動產生 scope 單元測試 整合測試 testware 骨幹、架構 完整系統 level 工具 方法論 test case 手動產生 自動產生 test driver / script 部份手寫 全自動 13

JUnit Framework 14

How to Test with JUnit? TestCase TestRunner Foo FooTest run 1..* exercise 1..* Foo FooTest test1 test2 … 15

Questions… 如何測試？ strategy 多少測試才夠？ coverage 流程？ process 工具影響？ medium is message 16

二、JUnit 基礎篇 Extreme Programmers test everything that could possibly break, using automated tests that must run perfectly all the time. --- Extreme Programming Installed

Design Goals of JUnit Easy to use Requires no more work than absolutely necessary to write a new test Leverages existing tests to create new ones Reusable fixtures to run different tests Retains values of tests over time Combines tests from various authors and run them together without fear of interference 18

Case Study 自動產生測試框架 Using Borland JBuilder 6.0 最簡單的完整測試實例 19

實例一 junit TestCase exercise 1..* Foo FooTest test1 test2 … 20

實例一：簡單的待測物 public class Money { private int fAmount; private String fCurrency; public Money(int amount, String currency) { fAmount = amount; fCurrency = currency; } public Money add(Money m) { return new Money(amount() + m.amount(), currency()); public int amount() { return fAmount; } public String currency() { return fCurrency; } 21

實例一：產生測試框架 [1/7] Borland JBuilder 6.0 22

實例一：產生測試框架 [2/7] Borland JBuilder 6.0 23

實例一：產生測試框架 [3/7] Borland JBuilder 6.0 24

實例一：產生測試框架 [4/7] Borland JBuilder 6.0 25

實例一：產生測試框架 [5/7] Borland JBuilder 6.0 26

實例一：產生測試框架 [6/7] Borland JBuilder 6.0 27

實例一：產生測試框架 [7/7] import junit.framework.*; 實例一：產生測試框架 [7/7] import junit.framework.*; public class MoneyTest extends TestCase { public MoneyTest (String s) { super(s); } protected void setUp() { } protected void tearDown() { } public void testAdd() { int val1 = 0; String val2 = "STRING0"; Money money = new Money(val1, val2); Money val1 = null /** @todo fill in non-null value */ ; Money moneyRet = money.add(val1); /** @todo: Insert test code here. Use assertEquals(), for example. */ } 28

Lesson Learned Code a little, test a little 29

實例二 TestCase TestRunner Foo FooTest run exercise 1..* test1 test2 … junit TestCase TestRunner run exercise 1..* Foo FooTest test1 test2 … 30

實例二：待測物，修正版 public class Money implements Cloneable { private int fAmount; private String fCurrency; public Money(int amount, String currency) { /*...*/ } public Money add(Money m) { if (m == null) return (Money) clone(); return new Money(amount() + m.amount(), currency()); } public Object clone() { /*...*/ } public boolean equals(Object obj) { /*...*/ } public int amount() { /*...*/ } public String currency() { /*...*/ } 31

實例二：測試碼 [1/2] import junit.framework.*; 實例二：測試碼 [1/2] import junit.framework.*; public class MoneyTest extends TestCase { public MoneyTest (String s) { super(s); } public void testNullAdd () { String curr = "NTD"; int val1 = 2002; Money money1 = new Money(val1, curr); Money money2 = null; // null value Money money3 = money1.add(money2); Assert.assertEquals(money1, money3); } 32

實例二：測試碼 [2/2] public void testSimpleAdd() { String curr = "NTD"; 實例二：測試碼 [2/2] public void testSimpleAdd() { String curr = "NTD"; int val1 = 2002; int val2 = 345; int val3 = val1 + val2; Money money1 = new Money(val1, curr); Money money2 = new Money(val2, curr); Money money3 = money1.add(money2); assertEquals(money3, new Money(val3, curr)); } 33

Success java -cp lib\junit.jar;. junit.swingui.TestRunner MoneyTest 34

Failure 1/2 35

Failure 2/2 36

Error 1/2 37

Error 2/2 38

Lessons Learned Generate & exercise test case(s) Verify Write test…() method(s) Verify Choose a Assert.assert…() Choose a TestRunner junit.textui.TestRunner junit.swingui.TestRunner Failure vs. error 39

三、JUnit 進階篇 The tests that you write in XP are isolated and automatic. --- Extreme Programming Explained

Case Study Fixture/context 管理 Test suite 與進入點 Grouping by tasks Configuration management Regression testing 41

實例三 TestCase TestRunner Foo FooTest run 1..* exercise 1..* test1 test2 junit TestCase TestRunner run 1..* exercise 1..* Foo FooTest test1 test2 … setUp() tearDown() 42

實例三：測試碼 [1/2] import junit.framework.*; 實例三：測試碼 [1/2] import junit.framework.*; public class MoneyTest extends TestCase { private Money money1, money2, money3; public MoneyTest(String s) { /*...*/ } protected void setUp() { String curr = "NTD"; int val1 = 2002; int val2 = 345; money1 = new Money(val1, curr); money2 = new Money(val2, curr); money3 = new Money(val1 + val2, curr); } 43

實例三：測試碼 [2/2] protected void tearDown() { } 實例三：測試碼 [2/2] protected void tearDown() { } public void testNullAdd() { Money money10 = money1.add(null); // null value assertEquals(money1, money10); } public void testSimpleAdd() { Money money10 = money1.add(money2); assertEquals(money3, money10); 44

Lessons Learned Fixture Each test…() method is isolated Overrides setUp() Overrides tearDown() Each test…() method is isolated 45

實例四 TestCase TestSuite TestRunner Foo FooTest test suite test case run junit TestCase TestSuite TestRunner run 1..* exercise 1..* Foo FooTest test1 test2 … test suite test case 46

實例四：測試碼 [1/3] import junit.framework.*; 實例四：測試碼 [1/3] import junit.framework.*; public class MoneyTest extends TestCase { private Money money1, money2, money3; public MoneyTest (String s) { /*...*/ } protected void setUp() { /*...*/ } public void testNullAdd() { /*...*/ } public void testSimpleAdd() { /*...*/ } 47

實例四：測試碼 [2/3] // [interactive mode] 實例四：測試碼 [2/3] // [interactive mode] // entry point of the whole test suite! /* // Version 1 public static Test suite() { TestSuite suite = new TestSuite(); suite.addTest(new MoneyTest("testNullAdd")); suite.addTest(new MoneyTest("testSimpleAdd")); return suite; } */ // Version 2 return new TestSuite(MoneyTest.class); 48

實例四：測試碼 [3/3] // [batch mode] // entry point of the whole test suite! 實例四：測試碼 [3/3] // [batch mode] // entry point of the whole test suite! public static void main(String args[]) { junit.textui.TestRunner.run(suite()); } 49

Lessons Learned Test case suite Write a suite() as the interactive mode entry point Write a main() as the batch mode entry point 50

實例五：Composite Pattern 51

實例五：before grouping FooTest test case test suite Foo exercise test 1 … test case test suite Foo 52

實例五：grouping by tasks test suite basic test FooTest random test Foo exercise Foo monkey test stress test 53

實例五：composite test suites public class Money_test extends TestCase { public MoneyTest (String s) { /*...*/ } public static Test suite() { TestSuite suite = new TestSuite(); suite.addTest(Money_test_basic.suite()); suite.addTest(Money_test_random.suite()); suite.addTest(Money_test_monkey.suite()); return suite; } public static void main(String args[]) { junit.textui.TestRunner.run(suite()); 54

55

Lessons Learned Grouping by configuration policy By tasks By persons By revisions Configuration management Naming scheme Process Scripts 56

小結：JUnit 使用四部曲 Fixture Exercise Verify Suite management Overrides setUp() and/or tearDown() Exercise Writes test…() Verify Uses Assert.assert…() Suite management Writes suite() and/or main() 57

其他主題 Reports TestResult Extensions for various application domains 58

四、軟體測試基礎 Program testing can be used to show the presence of defects, but never their absence! --- Edward Dijkstra

Why Verification/Testing is Hard? 理論限制 Intractable Undecidable 實務限制 成本 流程 人因 60

軟體測試方法論：分類 Scope Strategy Unit test Integration test System test Acceptance test Strategy Responsibility-based (black box) Implementation-based (white box) Hybrid (gray box) Fault-based 61

OO Testing Manifesto 觀察 信條 No silver bullet Every advance has a price Iterative & incremental Regression testing 信條 Fault models Test automation Test-effective process 62

Fault Models 重要性 正面: conformance-directed 反面: fault-directed exhausive testing 不可能，合理的 testing strategy 必須根據 fault models 的引導 正面: conformance-directed Feature sufficient 反面: fault-directed Fault efficient 63

五、推薦讀物 Tests are both a resource and a responsibility. --- Extreme Programming Explained

軟體測試通論 Edward Kit, Software Testing in the Real World: Improving the Process, Addison-Wesley/ACM Press, 1995. Boris Beizer, Black-Box Testing, John Wiley & Sons, 1995 Larry J. Morell and Lionel E. Deimel, Unit Analysis and Testing, SEI Curriculum Module SEI-CM-9-2.0, June 1992. http://www.sei.cmu.edu/publications/documents/cms/cm.009.html Hong Zhu, Patrick A. V. Hall, John H. R. May, “Software Unit Test Coverage and Adequacy,” ACM Computing Surveys, vol 29, no 4, 1997/12. 65

物件導向軟體測試 Robert V. Binder, Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley, 2000. 66

JUnit JUnit 網站: http://www.junit.org Test Infected: Programmers Love Writing Tests JUnit Cookbook JUnit: A Cook’s Tour 67

與 JUnit 有關的軟體開發流程 [Beck 1999] Kent Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley, 1999. Ron Jeffries, Ann Anderson, and Chet Hendrickson, Extreme Programming Installed, Addison-Wesley, 2001. 68

69