Request edit access
Finding Variation points in requirements with LLMs
Vorremmo studiare la capacità degli LLMs  di analizzare i requisiti del software, e abbiamo bisogno del vostro aiuto. 
  • Scegliete se usare chatGPT 3.5  oppure BING (in creative mode) 
  • Di seguito trovate 2 domande per il LLM. Quello che dovete fare è copiare la domanda (che comprende prompt e requisiti) e incollarla nella finestra di dialogo del chatbot, poi copiate e incollate la risposta nel modulo google. (Domanda 1 e domanda 2)
  • Poi, se possibile (ma se non avete tempo potete saltare questa parte) dovreste confrontare le risposte ottenute rispetto a un ground truth che vi forniamo (Domande 3,4,5,6)
  • Dite quale LLM avete usato!! per esempio nella mail: laura@chatGPT3.it
  • Infine inviate la risposta (in fondo al modulo). Raccolgo le mail solo per semplicità nella gestione delle risposte, possono essere di fantasia, ma corrette sintatticamente

Grazie,

Laura Semini e colleghi
Sign in to Google to save your progress. Learn more
Email *
Domanda1: copia il seguente testo sul LLM e incolla la risposta ottenuta 

You are an analyst charged with analyzing software requirements to define a software product line. Begin by listing all the features in the following requirements document. Then, identify the mandatory features and all points of variation within the document. To this end, apply the following tags to each feature or set of features: "mandatory" for individual mandatory features, "optional" for individual optional features, "or" for disjointed pairs or tuples of features, in which at least one must be present but two or more can coexist, "mutually exclusive" for pairs or tuples of features in which at least one must be present b are alternatives and cannot coexist, "excluded" for pairs of features that are not explicitly allowed together, and "requires" for pairs of features that depend on each other.

1.       After inserting a suitable coin, the user shall choose a beverage and  select the amount of sugar.

2.       The machine shall offer, as beverages, Coffee and Cappuccino or Tea.

3.       The machine shall always offer coffee.

4.       A ringtone possibly has to be played after beverage delivery.

5.       After the beverage is taken, the machine returns idle.

6.       The British market requires  tea and excludes any ring tone.

 *
Domanda2: copia il seguente testo sul LLM e incolla la risposta ottenuta 

 You are an analyst charged with analyzing software requirements to define a software product line. Begin by listing all the features in the following requirements document. Then, identify the mandatory features and all points of variation within the document. To this end, apply the following tags to each feature or set of features: "mandatory" for individual mandatory features, "optional" for individual optional features, "or" for disjointed pairs or tuples of features, in which at least one must be present but two or more can coexist, "mutually exclusive" for pairs or tuples of features in which at least one must be present b are alternatives and cannot coexist, "excluded" for pairs of features that are not explicitly allowed together, and "requires" for pairs of features that depend on each other.

1.       The system shall enable the search text to be entered on the screen.

2.       The system shall display all the matching products based on the search.

3.       The system possibly  notifies with a pop-up the user when no matching product is found on the search.

4.       The system shall allow a user to create his profile and set his credentials.

5.       The system shall authenticate user credentials to enter the profile.

6.       The system shall display the list of active and/or the list of completed orders in the customer profile.

7.       The system shall maintain customer email information as a required part of customer profile.

8.       The system shall send an order confirmation to the user through email.

9.       The system shall allow a user to add and remove products in the shopping cart.

10.   The system shall display various  shipping methods.

11.   The order shall be shipped to the client address or to an associated store, if the  click&collect service is available.

12.   The system shall enable the user to select the shipping method.

13.   The system may display the current tracking information about the order.

14.   The system shall display the available payment methods.

15.   The system shall allow the user to select the payment method for order.

16.   After delivery, the system may enable the users to enter their reviews or ratings.

17.   In order to publish the feedback on the purchases, the system needs to collect both reviews and ratings.

18.   The  click&collect service excludes  the tracking information service.

 *
Domanda 3a (true positives per le features della coffee machine): 

dite se il LLM ha trovato o meno le features elencate
Trovato
Coin insertion
Beverage selection
Sugar selection
Coffee
Cappuccino
Tea
Ringtone
Machine returns idle
Domanda 3b: falsi positivi. Indicare il numero di featutes trovate dal LLM che non sono in questa lista:

Coin insertion
Beverage selection
Sugar selection
Beverage selection
Coffee 
Cappuccino 
Tea
Ringtone
Machine returns idle
Domanda 4a (true positives per i punti di variabilità della coffee machine): 

dite se il LLM ha trovato o meno  i punti di variabilità elencati:
Trovato
Coin insertion (mandatory)
Beverage selection (mandatory)
Sugar selection (mandatory)
Coffee (mandatory)
Cappuccino, Tea (or)
Machine returns idle (mandatory)
Suitable coin (alternative coins)
British market (requires) tea
British market (excludes) ring tone
Ringtone (optional)
Domanda 4b: falsi positivi. Indicare il numero di punti di variabilità trovati dal LLM che non sono in questa lista:

Coin insertion (mandatory)
Beverage selection (mandatory)
Sugar selection (mandatory)
Coffee (mandatory) 
Cappuccino, Tea (or) 
Machine returns idle (mandatory)
Suitable coin (alternative coins)
British market (requires) tea
British market (excludes) ring tone
Ringtone (optional)

Domanda 5a (true positives per le features di eshop): 

dite se il LLM ha trovato o meno le features elencate
Trovato
Search functionality
Displaying matching products
Pop-up notification
User profile creation
Set user credentials
User authentication
Displaying active orders
Displaying complete orders
Order confirmation
Add to shopping cart
Remove from shopping cart
Display shipping methods
Selection of shipping method
Shipping option : client address
Shipping option : store
Click&collect service
Tracking information
Display payment methods
Selection of payment method
Publishing feedback
Enter feedback
Enter ratings
Enter reviews
Customer email information
Domanda 5b: falsi positivi. Indicare il numero di featutes trovate dal LLM che non sono in questa lista: 

Search functionality 
Displaying matching products 
Pop-up notification 
User profile creation 
Set user credentials
User authentication 
Displaying active orders 
Displaying complete orders 
Order confirmation
Add to shopping cart
Remove from shopping cart
Display shipping methods 
Selection of shipping method 
Shipping option : client address 
Shipping option : store
Click&collect service 
Tracking information 
Display payment methods 
Selection of payment method 
Publishing feedback 
Enter feedback
Enter ratings 
Enter reviews
Customer email information 

Domanda 6a (true positives per i punti di variuabilità di eshop): 

dite se il LLM ha trovato o meno i punti di variabilità elencati
Trovato
Search functionality (mandatory)
Displaying matching products (mandatory)
Pop-up notification (optional)
User profile creation (mandatory)
Set user credentials (mandatory)
User authentication (mandatory)
Displaying orders (mandatory)
Displaying active orders (or) Displaying complete orders
Order confirmation (mandatory)
Add to shopping cart (mandatory)
Remove from shopping cart (mandatory)
Displaying shipping methods (mandatory)
Selection of shipping method (mandatory)
Shipping option : client address (or) store
To store (requires) Click&collect
Enter feedback (optional)
Enter reviews (or) enter ratings
Publish feedback (optional)
Publish feedback (requires) enter review
Publish feedback (requires) enter ratings
Click&collect service (optional)
Click&collect service (excludes) tracking
Tracking information (optional)
Displaying payment methods (mandatory)
Selection of payment method (mandatory)
Customer email information (mandatory)
Riga 27
Domanda 6b: falsi positivi. Indicare il numero di punti di variabilità trovati dal LLM che non sono in questa lista: 

Search functionality  (mandatory)
Displaying matching products (mandatory)
Pop-up notification (optional)
User profile creation (mandatory)
Set user credentials (mandatory)
User authentication (mandatory)
Displaying orders (mandatory)
Displaying active orders  (or) Displaying complete orders 
Order confirmation  (mandatory)
Add to shopping cart (mandatory)
Remove from shopping cart  (mandatory)
Displaying shipping methods  (mandatory)
Selection of shipping method  (mandatory)
Shipping option : client address  (or)  store
 To store (requires) Click&collect
Enter feedback (optional)
Enter reviews (or) enter ratings
Publish feedback (optional)
Publish feedback (requires) enter review
 Publish feedback (requires) enter ratings
 Click&collect service (optional)
Click&collect service (excludes) tracking
 Tracking information (optional)
Displaying payment methods (mandatory)
Selection of payment method (mandatory)
Customer email information  (mandatory)

Submit
Clear form
Never submit passwords through Google Forms.
This form was created inside of Università di Pisa.

Does this form look suspicious? Report

Google Forms