From b12d31c9ae03b4b76d6157bae12437af408a2bb4 Mon Sep 17 00:00:00 2001 From: suspect15 <106546659+suspect15@users.noreply.github.com> Date: Fri, 1 Aug 2025 20:29:54 -0400 Subject: [PATCH 1/5] qa-oi update --- docs/.wip/WIP-Jr.md | 61 ---------------------- docs/general/teams/qa-team.md | 97 ++++++++++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 62 deletions(-) delete mode 100644 docs/.wip/WIP-Jr.md diff --git a/docs/.wip/WIP-Jr.md b/docs/.wip/WIP-Jr.md deleted file mode 100644 index 39e97e11..00000000 --- a/docs/.wip/WIP-Jr.md +++ /dev/null @@ -1,61 +0,0 @@ -WIP Page - Combining "Getting started as an achievement developer," "How to Become an Achievement Developer," and "Set Development Roadmap" into a sort-of Junior Dev doc. - -This document covers the basics of how achievements work and the steps one must take to get started with achievement development. - -**Anyone can become a RetroAchievements developer if they put in the effort!** - -- [Basic-Concepts](#basic-concepts) -- [Becoming a Junior Developer](#becoming-a-junior-developer) -- [Choosing a Game to Work On](#choosing-a-game-to-work-on) -- [Set Development Roadmap](#set-development-roadmap) - - [Initial Research](#initial-research) - - [Conceptual Design](#conceptual-design) - - [RAM Digging](#ram-digging) - - [Achievement Design](#achievement-design) - - [Pre-Release Steps](#pre-release-steps) - - [Release](#release) - - [Post-Release](#post-release) - -## Basic Concepts - -So how do RetroAchievements work? Technically, an achievement consists of a series of conditions that make up what we refer to as the `achievement logic`. When a game with an achievement set is launched in an emulator that supports RetroAchievements, the emulator constantly checks if the conditions of each achievement are met in the current frame of gameplay. When all conditions for a particular achievement are met, the emulator will stop checking the conditions of that achievement and award it to the player. - -The barebones process of creating achievement logic involves 3 steps: - -1. Finding a place in the game's RAM where the information needed for an achievement is stored. Some examples: current level or stage, health, timer, coins, points, ammo, holding a weapon, etc. -2. Creating conditions that must be met in order to award an achievement. -3. Test if the achievement is working as expected. - -However, this is often much easier said than done. There is a good chance that those steps will need to be repeated many times in order to refine the achievement logic to make it stable enough for players. - -## Becoming a Junior Developer - -If you would like to start developing achievements, the first thing you will need to do is read up on our Developer and Content Guidelines. It is also important to read the various pages in the Achievement Development category, but that can be done as you go. - -It is **strongly recommended** to join our Discord server so you can quickly and easily contact other achievement developers who can show you the ropes and help you along. Once you have joined the server, make sure you get verified by following the instructions in the `#welcome` channel. If you already know what game you'd like to make achievements for, you can go ahead and ask for the `jr-dev` role in the `#tech-support` forum channel by creating a post tagged as `Join Team` and pinging `QA-Maintainer`. Once you have the `jr-dev` role, you will have access to the `#jr-devs` channel, where you can talk to code reviewers and ask any development-related questions. - -## Choosing a Game to Work On - -After you have become a Junior Developer, you can decide on a game to work on. If you need an entry made for it, you can ask code reviewers to do so. They will need to open a forum topic for you anyway, assuming it doesn't already have one. You can now [claim it for achievement development](Claims-System). - -## Set Development Roadmap - -Once you have chosen your game and have claimed it for development, you need to plan your achievement set. You should know enough about the game to give it justice. The following are some tips on coming up with a plan: - -### Initial Research - -- **Read through the game manual:** Remember those? Game manuals often give insight into the game's story and mechanics. -- **GameFAQs**: Check the guides, FAQs, cheats, boards, etc. Soak up the info. -- **Longplays and Speedruns**: Seeing others play the game may give you ideas or show you something new. -- **Similar Sets**: Does a different port of the game have a set? What's it like? What would you change? - -### Conceptual Design - -- Start brainstorming your achievement list. How will you split up the progression achievements? What are the notable events in the game? Are there collectibles? Are there score benchmarks? What secrets/easter eggs are notable enough? Does the game have an internal reward system? -- Check our guide about [Achievement Design](/developer-docs/achievement-design) -- Check and test if your designs are possible. -- Figure out what data you will need to make the achievements. For example: achievements for collectibles will require the RAM addresses where their data is stored. - -## RAM Digging - -Once you have a plan, it is recommended that you post it in the game's forum topic. This will allow the community to give feedback and will give code reviewers an idea of where you're at from a design standpoint. **It is important to be open to receive feedback and suggestions!** diff --git a/docs/general/teams/qa-team.md b/docs/general/teams/qa-team.md index 8c0dcd08..87bef1fe 100644 --- a/docs/general/teams/qa-team.md +++ b/docs/general/teams/qa-team.md @@ -17,7 +17,7 @@ Current team members can be found on the [About Us](/general/about-us#quality-as ## QA's Responsibilties and Authorities -QA's responsibilities include maintaining inactive developer sets, ensuring all sets conform to the writing policy, add necessary achievement protections, repairing malfunctioning leaderboards, managing unstable sets, overseeing several sub-teams, onboarding people into the [Junior Developer Program](/developer-docs/jr-dev-rules), managing hashes, and maintaining content guidelines. +QA's responsibilities include maintaining inactive developer sets, ensuring all sets conform to the writing policy, add necessary achievement protections, repairing malfunctioning leaderboards, managing unstable sets, overseeing several sub-teams, managing hashes, and maintaining content guidelines. QA has the authority to fix any issues in sets with no active developers. QA may also fix issues in sets with active developers under the following processes: @@ -34,3 +34,98 @@ Developer must be contacted, QA may remediate if no response is received within - Adjust set progression types - Fix leaderboard or rich presence grammar, spelling, capitalization, or wording error - Repair leaderboard or rich presence malfunction + +# QA Operating Instructions + +### Team Member Activity Requirements + +Maintenance issues consistently arise and require an active QA team to respond and address them. Members of QA must meet the following minimum activity to remain on the team. Members who are unable to meet activity requirements are encouraged to step down or may be removed in order to ensure the team is staffed by active members. + +QA members must either accumulate 10 points in the previous month or meet 3-month lookback of 30 points to be considered active. Because some tasks requirement more time and effort, the following weighted task matrix is used to measure activity. + +10 point tasks: +- Repair a full set that lacks save/exploit/cheat protection +- Removing a set from the Outdated Version hub +- Fix set-wide instability issues on a set + +3 point tasks: +- Set up and complete a Player Compatibility Test request +- Removing a set from the No Dynamic RP hub +- Removing a set from the Missing Progression Typing hub +- Making minor repairs to set's leaderboard(s) per a reported problem +- An unspecified maintenance task commensurate with other 3 point tasks + +2 point tasks: +- Resolve an inactive developer ticket +- Removing a set from the Player-Input RP +- An unspecified maintenance task commensurate with other 2 point tasks + +1 point tasks: +- Close out a missable/progression tag report that requires updates to a set + +Members will track activity points as they close out issues in the #qa-log channel using the following format: + +``` +[action] [effected set] per [forum thread if applicable] [additional information as required] [activity point total for current month] +``` + +Example: "Updated progression markings in https://retroachievements.org/game/1391 per ⁠Trials of Mana Progression (discord link) 6 points in August" + +When a member reaches 10 points, subsequent issue closeouts may simply state "10+ points in [month]" + +## Task Processes +For issues not codified in these operating instructions, open a thread in the QA forum to discuss the issue and potential solutions. Do not act unilaterally. + +### Noncompliant Writing +- **Action if active dev** - Add the set to the [Meta|QA - Noncompliant Writing hub](https://retroachievements.org/hub/24397) for the Writing Team to track and update +- **Action if inactive dev** - Same as active dev +- **Authority** - Individual QA member + +### Confirmed Achievement Exploit +- **Action if active dev** - Notify via site message from QATeam account +- **Action if inactive dev** - Fix achievement and log in #qa-log or open thread and ping QA members to see if volunteer is available. If no available volunteer within 48 hours, demote achievement, log in #qa-log, and make post in official forum thread with QATeam account stating which achievement(s) were demoted and why +- **Authority** - Individual QA member + +### Rich Presence Maintenance +- **Action if active dev** - Notify via site message from QATeam account +- **Action if inactive dev** - Make required fixes, remove from Meta|QA hubs as appropriate, and log action and post backup of original RP in #qa-log +- **Authority** - Individual QA member + +### Leaderboard Issues +- **Action if active dev** - Notify via site message from QATeam account +- **Action if inactive dev** - Make required fixes, log action and back up logic in #qa-log. If leaderboard is unsalvagable, open thread to discuss deletion +- **Authority** - Individual QA member + +### Set-wide Exploit or Lack of Save Protection +- **Action if active dev** - Notify via site message from QATeam account and open a thread. May be accomplished by individual QA member +- **Action if inactive dev** - Open a thread, make fixes and post backup logic if able. See if other QA members are available to make repairs. If not, start vote to consider set demotion. If set is demoted, demote set, post notice in official forum topic with QATeam account and log action in #qa-log +- **Authority** - QA team majority vote to demote inactive developer set + +### Update to Content Guidelines +- **Action** - Make a thread with proposed update to discuss among QA team. Consider opening dev team discussion. Open a draft PR and post it in the thread for QA team proofreading. When ready to finalize, hold QA vote to approve adoption/merging of PR. +- **Authority** - QA team majority vote, usually informed by dev team inputs/poll + +### Conduct a Player Compatibility Test +- **Action if active dev** - Transfer request from QATeam account and offer to conduct test for active dev if desired +- **Action if inactive dev** - Link test hash to set, set to "Untested", and assign requestor the "tester" role. Confirm the tester has not unlocked any achievements and if so, direct them to reset the entire set before the test can commence. Inform tester that they must master or complete the set in softcore using only the test hash within a reasonable timeframe, usually 2 months, and to notify the QATeam account when the test is complete. Encourage tester to provide save states prior the unlock point for any incompatible achievements for potential updates. Validate that they mastered the set, then update the hash to "Compatible" or "Incompatible" if appropriate. + + In the event of incompatible achievements, QA may update achievements that were incompatible with save states provided by the tester at QA's discretion. + + If tester fails to complete the test for whatever reason, QA member who set up the compatibility test may terminate it at their discretion. +- **Authority** - Individual QA member + +### Progression, Win Condition, and Missable Tag Updates +- **Action if active dev** - Notify via site message from QATeam account +- **Action if inactive dev** - Make required fixes, log action in #qa-log +- **Authority** - Individual QA member + +### Other Issues Not Defined in this Operating Instruction +- **Action** - Open thread and discuss with QA team +- **Authority** - Individual QA member + +## Voting Process +Voting may be initiated by any QA member when required by these operating instructions or when desired to seek consensus. Votes should be conducted in respective discussion threads and the QA role shall be pinged upon creation of poll to inform all members. + +Members are encouraged to comment on their position if they have not already shared their perspective prior to voting. + +Voting duration shall be set to 3-days unless an exceptional circumstance as determined by the vote initiating member is determined. \ No newline at end of file From 54fde0d706e534478f757fc5b5ba0bfaaecdbb01 Mon Sep 17 00:00:00 2001 From: suspect15 <106546659+suspect15@users.noreply.github.com> Date: Sat, 2 Aug 2025 16:56:50 -0400 Subject: [PATCH 2/5] rewrite --- docs/general/teams/qa-team.md | 128 +++++----------------------------- 1 file changed, 16 insertions(+), 112 deletions(-) diff --git a/docs/general/teams/qa-team.md b/docs/general/teams/qa-team.md index 87bef1fe..26d61add 100644 --- a/docs/general/teams/qa-team.md +++ b/docs/general/teams/qa-team.md @@ -17,115 +17,19 @@ Current team members can be found on the [About Us](/general/about-us#quality-as ## QA's Responsibilties and Authorities -QA's responsibilities include maintaining inactive developer sets, ensuring all sets conform to the writing policy, add necessary achievement protections, repairing malfunctioning leaderboards, managing unstable sets, overseeing several sub-teams, managing hashes, and maintaining content guidelines. - -QA has the authority to fix any issues in sets with no active developers. QA may also fix issues in sets with active developers under the following processes: - -No advance coordination required: - -- Set-wide exploitation. QA must immediately contact the developer to hand off repair and repromotion responsibility -- Remove custom player input from rich presence to follow [the current policy](/developer-docs/rich-presence#rich-presence-custom-name-restriction) - -Developer must be contacted, QA may remediate if no response is received within 7 days for: - -- Demote set due to instability (generally 10 or more tickets) -- Fix achievement title or description grammar, spelling, capitalization, or wording error -- Demote achievement due to lack of protection or exploit -- Adjust set progression types -- Fix leaderboard or rich presence grammar, spelling, capitalization, or wording error -- Repair leaderboard or rich presence malfunction - -# QA Operating Instructions - -### Team Member Activity Requirements - -Maintenance issues consistently arise and require an active QA team to respond and address them. Members of QA must meet the following minimum activity to remain on the team. Members who are unable to meet activity requirements are encouraged to step down or may be removed in order to ensure the team is staffed by active members. - -QA members must either accumulate 10 points in the previous month or meet 3-month lookback of 30 points to be considered active. Because some tasks requirement more time and effort, the following weighted task matrix is used to measure activity. - -10 point tasks: -- Repair a full set that lacks save/exploit/cheat protection -- Removing a set from the Outdated Version hub -- Fix set-wide instability issues on a set - -3 point tasks: -- Set up and complete a Player Compatibility Test request -- Removing a set from the No Dynamic RP hub -- Removing a set from the Missing Progression Typing hub -- Making minor repairs to set's leaderboard(s) per a reported problem -- An unspecified maintenance task commensurate with other 3 point tasks - -2 point tasks: -- Resolve an inactive developer ticket -- Removing a set from the Player-Input RP -- An unspecified maintenance task commensurate with other 2 point tasks - -1 point tasks: -- Close out a missable/progression tag report that requires updates to a set - -Members will track activity points as they close out issues in the #qa-log channel using the following format: - -``` -[action] [effected set] per [forum thread if applicable] [additional information as required] [activity point total for current month] -``` - -Example: "Updated progression markings in https://retroachievements.org/game/1391 per ⁠Trials of Mana Progression (discord link) 6 points in August" - -When a member reaches 10 points, subsequent issue closeouts may simply state "10+ points in [month]" - -## Task Processes -For issues not codified in these operating instructions, open a thread in the QA forum to discuss the issue and potential solutions. Do not act unilaterally. - -### Noncompliant Writing -- **Action if active dev** - Add the set to the [Meta|QA - Noncompliant Writing hub](https://retroachievements.org/hub/24397) for the Writing Team to track and update -- **Action if inactive dev** - Same as active dev -- **Authority** - Individual QA member - -### Confirmed Achievement Exploit -- **Action if active dev** - Notify via site message from QATeam account -- **Action if inactive dev** - Fix achievement and log in #qa-log or open thread and ping QA members to see if volunteer is available. If no available volunteer within 48 hours, demote achievement, log in #qa-log, and make post in official forum thread with QATeam account stating which achievement(s) were demoted and why -- **Authority** - Individual QA member - -### Rich Presence Maintenance -- **Action if active dev** - Notify via site message from QATeam account -- **Action if inactive dev** - Make required fixes, remove from Meta|QA hubs as appropriate, and log action and post backup of original RP in #qa-log -- **Authority** - Individual QA member - -### Leaderboard Issues -- **Action if active dev** - Notify via site message from QATeam account -- **Action if inactive dev** - Make required fixes, log action and back up logic in #qa-log. If leaderboard is unsalvagable, open thread to discuss deletion -- **Authority** - Individual QA member - -### Set-wide Exploit or Lack of Save Protection -- **Action if active dev** - Notify via site message from QATeam account and open a thread. May be accomplished by individual QA member -- **Action if inactive dev** - Open a thread, make fixes and post backup logic if able. See if other QA members are available to make repairs. If not, start vote to consider set demotion. If set is demoted, demote set, post notice in official forum topic with QATeam account and log action in #qa-log -- **Authority** - QA team majority vote to demote inactive developer set - -### Update to Content Guidelines -- **Action** - Make a thread with proposed update to discuss among QA team. Consider opening dev team discussion. Open a draft PR and post it in the thread for QA team proofreading. When ready to finalize, hold QA vote to approve adoption/merging of PR. -- **Authority** - QA team majority vote, usually informed by dev team inputs/poll - -### Conduct a Player Compatibility Test -- **Action if active dev** - Transfer request from QATeam account and offer to conduct test for active dev if desired -- **Action if inactive dev** - Link test hash to set, set to "Untested", and assign requestor the "tester" role. Confirm the tester has not unlocked any achievements and if so, direct them to reset the entire set before the test can commence. Inform tester that they must master or complete the set in softcore using only the test hash within a reasonable timeframe, usually 2 months, and to notify the QATeam account when the test is complete. Encourage tester to provide save states prior the unlock point for any incompatible achievements for potential updates. Validate that they mastered the set, then update the hash to "Compatible" or "Incompatible" if appropriate. - - In the event of incompatible achievements, QA may update achievements that were incompatible with save states provided by the tester at QA's discretion. - - If tester fails to complete the test for whatever reason, QA member who set up the compatibility test may terminate it at their discretion. -- **Authority** - Individual QA member - -### Progression, Win Condition, and Missable Tag Updates -- **Action if active dev** - Notify via site message from QATeam account -- **Action if inactive dev** - Make required fixes, log action in #qa-log -- **Authority** - Individual QA member - -### Other Issues Not Defined in this Operating Instruction -- **Action** - Open thread and discuss with QA team -- **Authority** - Individual QA member - -## Voting Process -Voting may be initiated by any QA member when required by these operating instructions or when desired to seek consensus. Votes should be conducted in respective discussion threads and the QA role shall be pinged upon creation of poll to inform all members. - -Members are encouraged to comment on their position if they have not already shared their perspective prior to voting. - -Voting duration shall be set to 3-days unless an exceptional circumstance as determined by the vote initiating member is determined. \ No newline at end of file +QA's primary responsibilities include the following tasks: + +- Conduct hash management and administering [Player Compatibility Tests](/guidelines/content/player-compatibility-testing) +- Ensure set stability across the RetroAchievements library + - Repair or demote unstable or exploitable sets + - Resolve or close inactive developer tickets + - Ensure maintenance of leaderboards + - Ensure maintenance of Rich Presence +- Flag [noncompliant writing](guidelienes/writing-policy) for the writing team to address +- Ensure accurate type markings +- Address other miscellaneous quality issues +- Maintain RetroAchievements content guidelines + +QA has the inherent authority to rectify quality related issues in RetroAchievements sets. For any quality issue pertaining to content with an active developer, QA shall first contact the active developer to notify them of the issue and establish a resolution timeframe with one exception - QA may demote content that suffers from significant instability or exploitability prior to establishing two-way communication with an active developer. + +QA may add any sets to applicable Meta|QA hubs. \ No newline at end of file From dfa1d3f04d7f79cb8e36b35741cdbd7b1b8d406f Mon Sep 17 00:00:00 2001 From: suspect15 <106546659+suspect15@users.noreply.github.com> Date: Sat, 2 Aug 2025 17:00:53 -0400 Subject: [PATCH 3/5] typo fix --- docs/general/teams/qa-team.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/general/teams/qa-team.md b/docs/general/teams/qa-team.md index 26d61add..ff527367 100644 --- a/docs/general/teams/qa-team.md +++ b/docs/general/teams/qa-team.md @@ -25,7 +25,7 @@ QA's primary responsibilities include the following tasks: - Resolve or close inactive developer tickets - Ensure maintenance of leaderboards - Ensure maintenance of Rich Presence -- Flag [noncompliant writing](guidelienes/writing-policy) for the writing team to address +- Flag [noncompliant writing](guidelines/writing-policy) for the writing team to address - Ensure accurate type markings - Address other miscellaneous quality issues - Maintain RetroAchievements content guidelines From e2e4c54cc1d667051562aeacfb8a8c3cb2612abf Mon Sep 17 00:00:00 2001 From: suspect15 <106546659+suspect15@users.noreply.github.com> Date: Sat, 2 Aug 2025 17:02:35 -0400 Subject: [PATCH 4/5] typo fix --- docs/general/teams/qa-team.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/general/teams/qa-team.md b/docs/general/teams/qa-team.md index ff527367..13735983 100644 --- a/docs/general/teams/qa-team.md +++ b/docs/general/teams/qa-team.md @@ -25,7 +25,7 @@ QA's primary responsibilities include the following tasks: - Resolve or close inactive developer tickets - Ensure maintenance of leaderboards - Ensure maintenance of Rich Presence -- Flag [noncompliant writing](guidelines/writing-policy) for the writing team to address +- Flag [noncompliant writing](guidelines/content/writing-policy) for the writing team to address - Ensure accurate type markings - Address other miscellaneous quality issues - Maintain RetroAchievements content guidelines From ac10b3ab2053f08302379521419c86454380acc2 Mon Sep 17 00:00:00 2001 From: suspect15 <106546659+suspect15@users.noreply.github.com> Date: Sat, 2 Aug 2025 17:08:32 -0400 Subject: [PATCH 5/5] typo fix (last try) --- docs/general/teams/qa-team.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/general/teams/qa-team.md b/docs/general/teams/qa-team.md index 13735983..6ef92d7f 100644 --- a/docs/general/teams/qa-team.md +++ b/docs/general/teams/qa-team.md @@ -25,7 +25,7 @@ QA's primary responsibilities include the following tasks: - Resolve or close inactive developer tickets - Ensure maintenance of leaderboards - Ensure maintenance of Rich Presence -- Flag [noncompliant writing](guidelines/content/writing-policy) for the writing team to address +- Flag [noncompliant writing](/guidelines/content/writing-policy) for the writing team to address - Ensure accurate type markings - Address other miscellaneous quality issues - Maintain RetroAchievements content guidelines